Small Town Sysadmin
Free CJIS checklist

AI guidesBasicsFor everyone8 min read

What AI is good at, where it fails, and how to check its work

Hallucinations, knowledge cutoffs and confident wrong answers: what AI chatbots do well, where they go wrong, and a simple routine for checking anything they tell you.

Last checked:

AI chatbots can save real time, and they can also hand you a confident, well-written answer that’s completely wrong. Both come from the same place: they generate likely-sounding text (see how chatbots work). The trick is using them for the jobs they’re good at and checking them on everything else.

What they’re good at

These are jobs where a good first draft saves time and you can easily judge the result yourself:

  • Drafting and rewording. Emails, notices, policies, job postings, polite replies to difficult people.
  • Summarizing. Turning a long document or email thread you’ve provided into key points.
  • Explaining. “Explain this error message like I’m not an IT person,” or “What does this contract clause mean in plain English?” (then confirm anything important).
  • Formatting and structure. Turning notes into a checklist, a table, or step-by-step instructions.
  • Spreadsheet formulas. Describing what you want and getting a formula to try.
  • Brainstorming. Names, agenda ideas, questions to ask a vendor.
  • First-draft scripts and code for people who can read and test them (see AI for IT work).

Where they go wrong

Making things up. Chatbots can invent facts, statistics, quotes, studies, legal cases and links. OpenAI’s own help center warns that ChatGPT may make up citations and may sound very confident even when it’s wrong. Every vendor puts a similar warning in its product.

Sounding sure when they shouldn’t. In a September 2025 research paper, OpenAI explained one reason: the way models are trained and scored tends to reward a confident guess over admitting uncertainty, much like a multiple-choice test with no penalty for wrong answers.

Out-of-date knowledge. Each model has a knowledge cutoff. Without web search, it won’t know about recent events, new prices, current software versions or this month’s security patches, and it may not tell you it doesn’t know.

Losing track in long conversations. Very long chats and huge documents can push earlier details out of the model’s working memory, or make it pay less attention to them.

Agreeing with you. A leading question (“This is a scam, right?”) can nudge it toward the answer you seem to want. Ask neutrally.

Math, counting and dates. Assistants have improved a lot, especially when they can run code, but double-check any number that matters.

If a mistake would cost money, break the law, harm someone’s health, or fail an audit, an AI answer is a starting point for a conversation with a qualified person, not the final word.

How to check its work

A simple routine catches most problems:

  1. Ask for sources, then open them. A link the chatbot gives you might not exist, or might not say what it claims. Check that the page is real and supports the specific claim.
  2. Verify the specifics. Names, numbers, dates, prices, deadlines, version numbers, phone numbers and quotes are where errors hide. Check each one against the original source, such as the vendor’s site, the agency’s page or the actual document.
  3. Give it the source material. Instead of asking from memory, paste in or upload the document and tell it to answer only from that text and to quote the passage it relied on. Anthropic recommends this approach, along with letting the model say it doesn’t know.
  4. Give it permission to say “I don’t know.” Add something like “If you’re not sure, say so” to your prompt.
  5. Ask twice. Regenerate the answer or ask the same question in a new chat. If the answers disagree on a fact, treat that fact as unverified.
  6. Check the date. For anything time-sensitive, ask the assistant to search the web, then confirm on the official page.
  7. Test before you trust. A formula, script or set of instructions should be tried on a copy or a test machine first.

Good habits

  • Start a new chat for each new task, so old context doesn’t confuse things.
  • Give context: who you are, what you’re trying to do, and what “good” looks like.
  • Read the whole answer before acting on it, especially steps that delete, send or pay for something.
  • Keep a human name on anything that goes out the door. If you send it, you own it.
  • Don’t paste in anything sensitive (see Using AI safely).

Sources

Last checked October 3, 2026. AI products change quickly, so this guide is re-checked against official sources every month.

Back to the AI guides