For ten years I was on the vendor side of technology sales, which means I've watched these words get used two ways: precisely, by people who knew what they meant, and as decoration, by people who hoped you wouldn't ask. Then I started building products on top of language models myself, an HR assistant that runs on a self-hosted model among them, and had to learn what each term meant before I could choose one. This piece is that second education, written down.
Nothing here requires a technical background. I've tried to define each term the way I'd explain it across a table, and after each one I've added the practical reason it matters when you're evaluating a tool or a vendor. Skip to whichever ones you keep hearing.
The foundations
Artificial intelligence (AI)
The broad umbrella: any software that does something we'd normally associate with human thinking, like recognizing a face, understanding a sentence, or deciding what to do next. It's a category, not a specific technology. A spam filter from 2005 is technically AI. So is the chatbot you used this morning.
Why it matters: when a vendor says their product "uses AI," that tells you almost nothing. Ask what kind, and what it was trained to do.
Machine learning (ML)
The most common way of building AI today. Instead of a programmer writing rules ("if the email contains the word 'invoice', flag it"), you show the software thousands of examples and it works out the patterns itself. The software learns from data rather than being told what to do step by step.
Why it matters: because the system learned from data, its quality depends entirely on that data. A model trained on someone else's customers may not understand yours.
Neural network
A particular kind of machine learning system, loosely inspired by how neurons connect in a brain. It's made of layers of simple math units, each passing signals to the next. Individually they do almost nothing; stacked in enough layers, they can recognize speech, read handwriting, or write a paragraph. Nearly everything called "AI" in the news right now runs on neural networks.
Why it matters: mostly you don't need to care about the internals. But "deep learning" just means a neural network with many layers, so when you hear that phrase, it isn't a separate technology.
Large language model (LLM)
A very big neural network trained on enormous amounts of text to predict what words come next. That sounds trivial, but it turns out that predicting text well enough requires absorbing grammar, facts, reasoning patterns, and writing style. ChatGPT, Claude, and Gemini are all built on large language models.
Why it matters: an LLM is a text engine. It's very good at reading, summarizing, drafting, and answering from documents. It is not a database, not a calculator, and not a source of truth on its own. Products that treat it as one tend to disappoint.
The numbers people quote
Parameters
The internal settings a model learned during training, stored as numbers. Think of them as the knobs the training process turned to make the model's predictions better. A model with 7 billion parameters has 7 billion of these. A model with 400 billion has 400 billion. More parameters generally means more capability, but also more cost, more memory, and slower responses.
Why it matters: parameter count is a rough size label, not a quality score. A well-trained 8B model can beat a sloppy 70B one at a specific task. What you should ask is: does this model handle my use case well, and what does it cost to run?
Tokens
The units a language model reads and writes in. A token is roughly three-quarters of an English word: "understanding" might be two tokens, "the" is one. Models have a limit on how many tokens they can hold in mind at once (the "context window"), and cloud providers bill by the token.
Why it matters: this is how AI services are priced and why long conversations get expensive. If a tool quotes a per-token price, a 10-page document is roughly 7,000 tokens each time you send it. Small numbers add up fast.
Training vs. inference
Training is the one-time process of building a model from data. It costs millions of dollars and takes weeks on specialized hardware. Inference is using the finished model to answer a question. That's what happens every time you type a prompt. Training happens once, at the vendor; inference happens constantly, and that's what you pay for.
Why it matters: "we trained our own model" is a big claim and rarely true for small vendors. Most products run inference on a model someone else trained, which is fine, but you should know whose.
Open vs. closed, and the term that gets misused
Closed model (proprietary)
A model you can use but not inspect or download. You send your request to the vendor's servers, they run it, they send back the answer. GPT-4, Claude, and Gemini are closed. You never see the model itself.
Why it matters: convenient and usually the strongest performers, but your data goes to a third party, you depend on their pricing and availability, and if they discontinue the model, you migrate.
Open weight model
A model whose parameters (the "weights") are published for anyone to download and run. You can put it on your own server, run it without an internet connection, and no data leaves your building. Llama, Mistral, and Qwen are common examples. What's not included is the training data or the full recipe used to build it, so you can use the model but you couldn't recreate it.
Why it matters: this is the option when privacy or cost is the constraint. HR documents, legal files, patient records, anything you can't send to a cloud API. It trades some raw capability for control.
Open source model
Here's where the terminology gets sloppy. Strictly, "open source" means everything is published: the weights, the code, and the training data, under a license that lets you do anything with it. Very few models meet that bar. Most models marketed as "open source" are actually open weight: the weights are free, but the data isn't shared and the license may restrict commercial use above a certain company size.
Why it matters: if a vendor says "open source," read the license before building on it. Some "open" licenses prohibit exactly what you'd want to do. The word you usually want is open weight, and the question you should ask is "what does the license permit?"
The practical split: closed models when you need the best possible output and your data isn't sensitive. Open weight models when data can't leave your control, when volume makes per-token pricing painful, or when you need to run without internet. Most businesses end up with both, matched to different jobs.
Terms from the product side
Prompt
Whatever you type into the model. Also, more broadly, the instructions a product sends to the model behind the scenes on your behalf. Most AI products are a model plus a carefully written set of prompts that shape what it does.
Why it matters: a surprising number of "AI products" are a standard model with good prompts and a nice interface. That's not a criticism, but it does mean the differentiator is often the prompts and the workflow, not the model.
Retrieval-augmented generation (RAG)
A technique for making a model answer from your documents instead of from its general training. When you ask a question, the system first searches your files for the relevant passages, then hands those passages to the model along with your question. The model writes its answer from what it was given. This is how most "chat with your documents" tools work, including HR assistants and support bots.
Why it matters: if you want an AI tool grounded in your own policies, contracts, or product manuals, RAG is what you're buying. Ask whether it cites where the answer came from. Good RAG systems do; it's how you verify instead of trust.
Fine-tuning
Taking an existing model and training it a little more on your own examples so it picks up your tone, format, or specialized vocabulary. It's far cheaper than training from scratch but more involved than prompting. Think of it as sending the model to a short course rather than through the whole school.
Why it matters: often oversold. For most business needs, good prompts plus RAG get you there without fine-tuning. It's worth it when you need a very specific style at high volume.
Hallucination
When a model states something false with complete confidence. It's not lying; it's predicting plausible text, and sometimes plausible text is wrong. A model can invent a case citation, a product feature, or a statistic that sounds entirely reasonable.
Why it matters: this is why "keep a human on the send button" isn't paranoia. Any AI output going to a customer, a regulator, or a contract needs a person reading it first. Systems with RAG and citations reduce this substantially; nothing eliminates it.
AI agent
A model that doesn't just answer but takes actions: searches the web, reads a file, updates a record, sends an email, then decides what to do next based on the result. "Agentic" is the adjective. The difference from a chatbot is that an agent has tools and a loop: it can try something, see what happened, and try again.
Why it matters: agents are where the real automation lives, and also where the real risk lives, because they act. When a vendor says "agent," ask what actions it can take, whether a human approves them, and what happens when it gets something wrong.
What to actually ask a vendor
With the vocabulary in place, the questions get simple. Which model is under the hood, and is it closed or open weight? Where does my data go when I use this? Does it answer from my documents, and does it show me where the answer came from? What does it cost at my volume, in tokens or otherwise? And what can it do on its own versus what it drafts for a person to approve?
A vendor who can answer those five plainly is one you can work with. One who retreats into "it's proprietary" or "our AI is very advanced" hasn't told you anything, and probably knows it.
Evaluating an AI tool and not sure what the vendor is actually offering? I help businesses sort the practical from the hype, starting with a look at where your team's hours actually go.
Start a conversation