67 terms, grouped by what you are actually trying to decide.
Every term on this page, alphabetically. Each one links straight to its definition. If you arrived here from a proposal with one word you did not recognise, start here.
Reading this before a vendor call is the cheapest preparation available. If you would rather see the terms applied to a real decision, the local versus cloud comparison and the models and hardware sizing guide both use this exact vocabulary.
The eight words that carry most of the confusion. Get these right and the rest of the page reads easily.
The kind of AI behind ChatGPT-style tools: a program trained on a very large amount of text that predicts and generates language. It reads what you write and writes back.
Why it matters to you: Everything else on this page is either a part of an LLM, a way of running one, or a way of controlling one.
One specific trained AI, identified by publisher, name and version, for example Qwen3-30B or Llama 4 Scout. A model is a file (or set of files) you load and run, not a service you subscribe to.
Why it matters to you: You can pin an exact model version and keep it frozen, so an answer that was correct last quarter is still reproducible this quarter.
The numbers inside a trained model that encode what it learned. When people say they downloaded a model, they mean they downloaded its weights.
How many of those numbers a model has, counted in billions and written as 8B, 30B, 70B. More parameters generally means more capability, more memory needed and slower generation.
Why it matters to you: Parameter count is the first number that decides which hardware you need. It is a size, not a quality score: a well-trained 30B model can beat a poorly trained 70B one.
A model whose weights the publisher has released for download, so anyone can run it on their own hardware. Qwen, Llama, Gemma, DeepSeek and Mistral families are open weight.
Why it matters to you: Open weight is the single property that makes local AI possible at all. If the weights are not published, the model can only ever run on somebody else's server.
Running a trained model to get an answer. Distinct from training, which is the far more expensive process of creating the model in the first place.
Why it matters to you: You almost certainly want to do inference on your own hardware and let somebody else pay for the training.
The text you send the model: your question plus any instructions and pasted material. Everything the model knows about your immediate task arrives through the prompt.
The chunk of text a model reads and writes, roughly three quarters of an English word. A 1,000 word document is around 1,300 tokens.
Why it matters to you: Tokens are the unit that cloud vendors bill and that local hardware limits. Once you think in tokens, both pricing and sizing stop being mysterious.
How much text the model can hold in view at once, measured in tokens. It covers the system prompt, the conversation so far, any retrieved documents and the answer being written.
Why it matters to you: The context window sets the largest single document the model can read end to end. When it fills, the earliest text falls out of view, which is why long chats start to forget.
A setting, usually between 0 and 1, controlling how varied the model's wording is. Near 0 the model plays it straight; higher values produce more variety.
Why it matters to you: Set it near 0 for extraction, classification and anything involving figures. Save the higher values for drafting and idea generation.
The vocabulary of the specification sheet: what makes a model fit on a machine, and what makes it feel fast.
Compressing a model's weights to fewer bits each (Q8, Q5, Q4) so it needs less memory and runs faster. Quality falls gradually as the bit count drops rather than collapsing at a threshold.
Why it matters to you: Quantisation is what lets a large model fit on affordable hardware. It is the main reason a serious model can run on a workstation rather than a rack of accelerators.
The dedicated memory on a graphics card. The model weights, the context and the KV cache all have to fit in it for the model to run at full speed.
Why it matters to you: VRAM is usually the binding constraint on which models you can run. Size the VRAM first, then choose the model, not the other way round.
A widely used single-file format for packaging quantised model weights so a local runtime can load them. Most downloadable local models are distributed as GGUF files.
The software that loads a model and answers requests: Ollama for simplicity, vLLM or SGLang for high-throughput multi-user serving, llama.cpp for lean single-machine setups.
Why it matters to you: The runtime choice mostly decides how many people the same hardware can serve at once.
How fast the model writes, measured in generated tokens each second. Around 15 to 20 tokens per second reads at roughly comfortable reading speed for one user.
Why it matters to you: It is the headline speed number, but on its own it describes one person using the machine alone. Ask for it at your real concurrency.
The delay between pressing send and the first word appearing, while the model reads your prompt and any retrieved documents.
Why it matters to you: This is what users actually perceive as fast or slow, far more than total generation speed. A long document in the prompt lengthens it.
A store of the model's working state for the conversation so far, so it does not have to re-read everything for each new word. It lives in VRAM and grows with context length.
Why it matters to you: The KV cache is the memory cost people forget when sizing. Long contexts and many simultaneous users both enlarge it, and when it will not fit, speed drops sharply.
A model built from many specialist sub-networks where only a few activate for any given token. A 30B MoE model may use only around 3B parameters per token.
Why it matters to you: MoE gives large-model quality at a much lower running cost. The catch is that all the weights still have to be in memory even though only a fraction is used at a time.
The total wait for a complete answer, from the moment you send to the moment the last word lands.
The total volume of tokens a server produces per second across every user it is serving at that moment.
Why it matters to you: Throughput, not single-user speed, is what determines whether one server can carry a department.
How many people can use the system at the same time before each of them notices it slowing down.
Why it matters to you: Concurrency is the number to size against. A demo that flies for one person can crawl at twelve, and the gap is invisible in any benchmark score.
Serving several users' requests together in one pass through the model, which is how a good runtime keeps throughput high as concurrency rises.
How a general model comes to know your products, your policies and your documents. This is where most first projects succeed or fail.
Connecting the model to your own documents so that, before answering, it searches them and reads the most relevant passages. The answer can then cite the exact source.
Why it matters to you: RAG is the correct home for facts. Correcting a wrong answer means replacing a document, which anyone can do, rather than retraining a model.
Numeric fingerprints of text produced by a small model, arranged so that passages with similar meaning end up with similar numbers.
Why it matters to you: Embeddings are why RAG finds the right paragraph when the user's words do not match the document's words at all.
The store that holds embeddings and finds the closest matches very quickly. pgvector, Qdrant and Chroma are common choices.
Why it matters to you: It is the search index behind RAG, and it holds a searchable copy of your documents, so it belongs in your backup routine and inside your access controls.
Splitting documents into passages small enough to retrieve precisely but large enough to still make sense on their own.
Why it matters to you: Bad chunking is the most common cause of a RAG system that retrieves the right document but the wrong paragraph.
Further training an existing model on your own examples to change its habits: its house style, its output shape, its label vocabulary.
Why it matters to you: Fine-tuning changes behaviour, not knowledge. A fact baked into a fine-tune cannot be corrected without retraining, which is why facts belong in RAG.
A low-cost fine-tuning method that trains a small add-on layer instead of the whole model, so it needs far less hardware and time and can be switched on and off.
Why it matters to you: LoRA is what makes fine-tuning affordable at business scale, but it still has to be redone when you move to a newer base model.
The standing instruction given to the model before every conversation: its role, its tone, its boundaries and the exact format its answers must take.
Why it matters to you: More of the difference between a useless and a useful deployment lives here than anywhere else, and it is the cheapest thing on this page to change.
Requiring the model to answer only from supplied source material, and to say it does not know when the sources do not cover the question.
A repeatable automated test of answer quality, run after every model, prompt or document change to check that nothing got worse.
Why it matters to you: Without evals you have no way to tell an improvement from a regression, and every upgrade becomes a gamble.
A fixed list of real questions from your own business, each with the answer a competent colleague would give. The eval scores the system against it.
Why it matters to you: Thirty to a hundred genuine cases is usually enough to catch regressions. Building it is the highest-value unglamorous work in an AI project.
A public leaderboard score comparing models on standard test sets.
Why it matters to you: Useful for drawing up a shortlist, useless as acceptance criteria. It says nothing about your invoices, your policies or your file formats.
The vocabulary of automation: the terms that appear once the AI stops answering questions and starts completing tasks.
A model given a goal rather than a single question. It plans its own steps, calls tools, checks results and loops until the goal is met or it stops.
Why it matters to you: An agent can complete a task end to end, which also means a mistake can propagate further than a wrong sentence. Approval gates matter more here than anywhere.
The model asking software to do something it cannot do itself, such as reading a stock level or querying a database, then continuing with the returned answer.
Why it matters to you: Tool calling is how the AI reaches live data. Anything that changes minute to minute should come through a tool, never from the model's memory.
An open standard for connecting a model to tools and data sources through a consistent interface, so each new integration is a configuration rather than a bespoke build.
Why it matters to you: MCP reduces lock-in: connectors written against the standard keep working when you change model or runtime.
The framework that decides step order, retries failures and records what happened. LangGraph, n8n and similar tools fill this role.
Why it matters to you: An agent without orchestration is a demo. With it, it is a business process that can be audited and rerun.
A fixed, predefined sequence of steps with the model doing specific jobs inside it, for example read the invoice, extract the fields, match the purchase order, queue for approval.
Why it matters to you: For repeatable high-volume work, a workflow is more predictable, cheaper and easier to audit than an agent. Most production value sits here, not in open-ended agents.
A model driving a graphical interface directly, reading the screen and moving a pointer, to operate software that offers no API.
Why it matters to you: It unlocks legacy systems that cannot be integrated any other way, at the cost of being slower and more brittle than a real integration.
A model that accepts more than plain text as input, typically images or audio alongside it.
A model that reads images, scans and PDFs, so it can pull figures off a photographed invoice, a stamped delivery order or a handwritten form.
Why it matters to you: Most real business documents arrive as scans rather than clean text, so a VLM is often the difference between a document project working and stalling.
Speech to text. Whisper is the widely used open model, and it runs locally, so recordings never leave the building.
Why it matters to you: Consultation notes, meeting minutes and site reports are all transcription problems before they are AI problems.
A model trained to work through a problem in explicit steps before answering. Slower and more costly per answer, stronger on multi-step logic.
Why it matters to you: Worth the wait on genuinely hard analysis. Wasted on routine drafting, summarising and extraction.
Constraining the model to return data in a fixed shape such as JSON with named fields, so another system can consume it reliably.
Why it matters to you: This is what turns AI output into something your ERP or accounting system can accept without a human retyping it.
The terms that decide whether the system can be signed off. Most of these describe controls that live outside the model, which is exactly why they still hold when the model is wrong.
When a model states something false with the same confidence it states something true. It is a property of how these models generate text, not a bug awaiting a fix.
Why it matters to you: Design for it rather than hoping it away: ground answers in retrieved documents, show citations, and keep a human on anything irreversible.
Showing, next to each answer, the exact source passages it was drawn from, so a reader can check it in seconds.
Why it matters to you: Citations convert an unverifiable claim into a checkable one. They do more for adoption than any accuracy percentage.
The rules surrounding the model: what it may read, what it may write to, which actions need approval, and what it must refuse.
Why it matters to you: Guardrails sit outside the model, so they keep working on the day the model gets something wrong.
A named person approves an action before it takes effect. Standard practice on every irreversible step: paying, posting to a ledger, sending to a client, deleting.
Why it matters to you: It is the control that turns an interesting pilot into something a finance or compliance lead will actually sign.
A single control that halts every running agent at once, putting a hard ceiling on how much a bad run can do.
A document, email or web page containing text crafted to give the model new instructions, for example telling it to ignore its rules and send a file somewhere.
Why it matters to you: It is the distinctive security risk of AI systems. The defence is architectural: treat all retrieved content as data rather than instructions, and keep irreversible actions behind human approval.
A tamper-evident record of every prompt, every retrieved document and every action, with the user and the timestamp attached.
Why it matters to you: This is the artefact an auditor, a regulator or your own investigation actually asks for. Retrofitting it later is painful, so specify it up front.
Deciding who may see which documents and use which tools by job role rather than person by person.
Why it matters to you: Without RBAC, a RAG index quietly becomes the fastest route for any employee to read the payroll folder.
A machine with no network connection to the internet at all. Models and updates are applied manually through controlled media.
Why it matters to you: The strongest privacy posture available, because there is no route out even by accident. It costs convenience in updates, so it is chosen deliberately, not by default.
Which country your data is physically stored and processed in. On a cloud service this is set by the vendor's available regions; on your own hardware it is set by where the machine sits.
Which country's laws and authorities can reach your data. Related to residency but not the same: a foreign-owned operator can be subject to foreign law even when the servers sit locally.
Why it matters to you: For government and regulated work this, rather than performance, is often the whole reason a local deployment is chosen. See the guidance for public sector buyers.
Malaysia's Personal Data Protection Act 2010, which governs how personal data is collected, processed, stored and transferred.
Why it matters to you: The practical consequence for AI is simple: personal data you never transmit is personal data you never have to justify transmitting.
Stored files and the traffic between machines are both encrypted, so a stolen disk or a tapped cable yields nothing readable.
Staff using consumer AI tools on personal accounts, outside any corporate agreement, log or policy.
Why it matters to you: This is the leak most organisations actually suffer. The reliable fix is giving people an internal tool that is fast and always available, so there is no reason to go elsewhere.
The words that appear in the proposal and the finance review rather than the technical spec.
A local build is capital expenditure: bought once, owned, depreciated over its life. A cloud subscription is operating expenditure: a recurring cost that never ends.
Why it matters to you: Which one your finance lead prefers is a real question worth settling before anyone writes a proposal, because it changes the recommendation.
The full cost across the life of the system: hardware, setup, electricity, maintenance, and the eventual refresh, not just the purchase price.
Why it matters to you: TCO is the only fair basis for comparing a one-time build against a subscription. Compare purchase price alone and you will reach the wrong answer either way.
The point at which the cumulative cost of the subscription you replaced equals the one-time cost of the build. After it, the local system runs for close to the price of electricity.
Why it matters to you: VYROX builds typically break even against the subscriptions they replace within 6 to 14 months, backed by a Break-Even Guarantee.
Cloud AI charges either a fixed fee per user per month, a fee per token consumed, or both. Local AI charges neither once the build is paid for.
Why it matters to you: Per-seat pricing punishes rolling the tool out widely, which is the opposite of what you want once it starts working.
How hard it would be to leave: proprietary formats, non-portable connectors, prompts tuned to one vendor's model, data you cannot export.
Why it matters to you: The honest test is to ask a supplier what leaving looks like. Standard open-source tooling and open standards such as MCP keep the answer short.
Knowing exactly which model version, from which publisher, under which licence is running, and being able to freeze it under change control.
Why it matters to you: Auditors and security reviewers ask this. It is also what makes an answer from last quarter reproducible today.
The terms attached to a downloadable model. They vary: some are true open-source licences such as Apache 2.0 or MIT, others are community licences with their own conditions on commercial use, scale or redistribution.
Why it matters to you: Open weight does not automatically mean unrestricted commercial use. Check the licence of each specific model before it goes into production.
The written commitment on response times and support. For a local build it covers the supplier's response, not the model's uptime, because the machine is yours.
A deliberately small first deployment, usually one department or one workstation, run on real work to prove both the quality bar and the cost case before scaling.
Why it matters to you: A pilot with a golden set and a measured baseline tells you something. A pilot without them just tells you people enjoyed trying it.
Four pairs of words that get used interchangeably and should not be. Picking the wrong one of each is the most expensive kind of mistake in a first AI project, because it is usually only discovered after the build.
Swipe to see all columns
| Prompting | RAG | Fine-tuning / LoRA | |
|---|---|---|---|
| What it changes | The instructions the model follows | What the model can look up before answering | The model's own habits and output style |
| Right for | Tone, format, length, refusal rules | Your products, prices, policies and documents | A very specific in-house style or label set |
| Fixing a mistake | Edit the instruction, live the same hour | Replace the document in the index | Retrain, which needs new examples and time |
| Can it cite a source | No | Yes, that is the point of it | No |
| Typical effort | Hours | Days to a few weeks, depending on how tidy the documents are | Weeks, and it must be redone on a newer base model |
Facts belong in RAG. Habits belong in a fine-tune. Instructions belong in the prompt. Most first projects need prompting plus RAG and no fine-tuning at all, so if a supplier opens with fine-tuning before seeing your documents, ask them why.
Swipe to see all columns
| Open weight | Open source | Closed API model | |
|---|---|---|---|
| Can you download it | Yes | Yes | No |
| Can you run it offline | Yes | Yes | No |
| Licence terms | Vary by model: some permissive, some with commercial conditions | A recognised open-source licence such as Apache 2.0 or MIT | Terms of service, changeable by the vendor |
| Training data published | Usually not | Sometimes, in the strictest sense of the term | No |
| What to check first | The specific model's licence, before production | Nothing beyond the licence text | Retention, sub-processors and processing region |
Open weight is about access, open source is about permission. Every open-source model is open weight; not every open-weight model is open source. The model and hardware guide lists which families are in current use.
Swipe to see all columns
| Local / on-premise | Private cloud (your tenancy) | Public cloud API | |
|---|---|---|---|
| Where the model runs | Hardware you own, on your premises | A GPU instance inside your own cloud account | The vendor's servers |
| Works with the internet down | Yes | No | No |
| Recurring cost | Electricity and maintenance | Per hour the instance runs | Per seat and per token, forever |
| Who operates it | You, with supplier support | Your team, in full | The vendor |
| Best when | Data is sensitive and usage is steady | You already run cloud well and demand is spiky | Usage is light or exploratory |
All three can be described as private by somebody. Only one of them keeps working when the connection drops, and only one of them ends the recurring bill. The case for running AI locally works through the trade-offs in full.
Swipe to see all columns
| Chatbot | Workflow automation | Agent | |
|---|---|---|---|
| What it is given | A question | A fixed sequence of steps to run | A goal, with the steps left open |
| Who decides the steps | Nobody, it answers and stops | You, in advance | The model, at run time |
| Predictability | High | High | Lower, by design |
| Easiest to audit | Yes | Yes | Needs orchestration and logging to be auditable at all |
| Where the business value usually sits | Everyday drafting and internal search | High-volume repeatable document and data work | Open-ended research and multi-system tasks |
The word agent sells better than the word workflow, which is why proposals are full of agents. For repeatable high-volume work the workflow is usually the better engineering answer. The solutions by department page shows which shape suits which job.
A glossary is only useful if it changes a conversation. These are the questions the terms above are for, and the shape of an answer worth trusting.
A single-user speed figure is close to meaningless for an office. Ask for the number with the number of people you actually have using it at once, and ask for time to first token separately, because that is what people perceive as speed.
If the answer is a fine-tune and the underlying problem is that the model does not know your prices or policies, the design is wrong. Facts change, and a fine-tuned fact cannot be corrected without retraining.
The right answer contains the words golden set and eval. If the answer is that you will notice if it gets worse, you will not, and neither will they.
RBAC has to sit on the retrieval layer, not in the prompt. A rule that says the model should not mention payroll is not an access control, it is a request.
Prompt, retrieved documents, action taken, user and timestamp. Anything less and you cannot reconstruct what happened when someone eventually asks.
Ask which model version is pinned, which parts are standard open-source tooling, and how the documents and vector database come out. A short, specific answer means low lock-in. A vague one is the answer.
Costs, tiers and payback figures are not defined here on purpose, because they are numbers rather than vocabulary. They live on the pricing page, and public sector procurement language is covered separately in the guidance for government buyers.
The same answers as above, written out at the length you would want if you were about to repeat them to a colleague.
More context for each of these sits in the wider resources and buyer guides, or start from the local AI overview if this is your first visit.
Book a free 45-minute Local-AI Audit. We measure your current cloud spend, spec the exact build, and give you the costed break-even date in writing, no obligation.
No deck pitch. Just engineers sizing your build.
Free Local-AI audit