Small Town Sysadmin
Free CJIS checklist

AI guidesBasicsFor everyone7 min read

What AI is, and how chatbots like ChatGPT actually work

A plain-English explanation of AI chatbots: what a large language model is, why it sounds so confident, and the handful of terms you'll keep running into.

Last checked:

“AI” gets stuck on everything from spam filters to toasters, so it helps to be specific. When people in the office talk about AI right now, they almost always mean a chatbot like ChatGPT, Claude, Gemini or Microsoft Copilot: a box you type (or talk) into that writes back in fluent, natural language.

You don’t need to understand the math to use these tools well. But knowing roughly how they work explains both why they’re so useful and why they sometimes get things badly wrong.

The short version

A chatbot is built on a large language model, usually shortened to LLM. An LLM is a computer program trained on an enormous amount of text so that it gets very good at one job: predicting what text should come next.

When you ask a question, the model doesn’t look the answer up in a database. It writes a response one small piece at a time, each time choosing a likely next piece based on everything it learned in training and everything in your conversation so far. Do that fast enough, with a big enough model, and the result reads like a knowledgeable person typing.

That’s the whole trick, and it’s why two things are true at once:

  • It’s remarkably good at language: drafting, summarizing, rewording, explaining, translating and brainstorming.
  • It can produce something that sounds exactly right and isn’t, because sounding right is what it’s built to do. The next guide covers this in detail.

How a chatbot gets made

Training. The vendor feeds the model a huge collection of text (books, websites, code and other material) and has it practice predicting missing or next words billions of times. Along the way it picks up grammar, facts, writing styles and patterns of reasoning, all stored as numbers rather than as a list of facts it can look up.

Tuning. The raw model is then trained further so it follows instructions, holds a conversation, and declines certain requests. This is where each company’s assistant develops its own personality and rules.

The cutoff. Training stops at some point, so the model’s built-in knowledge has a knowledge cutoff date. Anything that happened after that date is unknown to it unless the chatbot has a tool, like web search, to go and find it.

The add-ons that make chatbots more useful

Modern assistants are more than the bare model. Depending on the product and plan, they can:

  • Search the web and cite pages, which helps with recent events (though you should still open the sources).
  • Read files you upload, like a PDF, spreadsheet or screenshot.
  • Connect to your other accounts, such as email, calendar or cloud storage, when you or your IT admin allow it.
  • Remember things across conversations, if a memory feature is turned on.
  • Take actions on your behalf, like running code, filling out a form or working through a multi-step task. Products that do this are often called agents.

Each add-on is useful, and each is also a new way for information to flow in or out. That matters for safety, covered in Using AI safely.

Terms you’ll keep hearing

Model. The trained program itself. Vendors release new models every few months and give them names like GPT, Claude Sonnet or Gemini Pro. The chatbot app is the product you use; the model is the engine inside it.

Prompt. Whatever you type or say to the chatbot. Clear, specific prompts with context get better answers.

Token. The small chunks of text a model reads and writes. OpenAI’s rule of thumb is that one token is about three-quarters of an English word, so 100 tokens is roughly 75 words. Usage limits and pricing are often measured in tokens.

Context window. How much text the model can consider at once: your conversation, any files you’ve shared, and its own replies. Anthropic describes it as the model’s working memory. When a very long conversation exceeds it, earlier material drops out or gets less attention, which is one reason starting a fresh chat for a new task helps.

Hallucination. When a model states something false as if it were fact, such as a made-up statistic, quote or court case. Researchers also call this confabulation.

Knowledge cutoff. The date the model’s training data ends.

Agent. An AI system that can take actions, not just write text.

What it isn’t

A chatbot isn’t a search engine, a database, or a person. It doesn’t know when it’s wrong unless something tells it, it can’t vouch for its own sources, and it has no stake in whether your printer gets fixed. Treat it like a fast, well-read assistant whose work you always check.

Sources

Last checked October 3, 2026. AI products change quickly, so this guide is re-checked against official sources every month.

Back to the AI guides