How to check AI before using its answers

The most dangerous AI answer isn’t the absurd one.

You catch the absurd one. You laugh, you screenshot it, you move on.

The dangerous one is the answer that is 90% right, well written, perfectly formatted — and wrong exactly where it matters. A deadline off by a week. A regulation that was amended. A function that doesn’t exist. A number that looks plausible because it was built to look plausible.

That is the real fear behind “What if I trust a wrong answer?” Not that AI makes mistakes. Every tool does. The fear is that it makes them in the same voice it uses when it’s right.

Fluency is not accuracy

A language model is very good at producing text that sounds like a correct answer. Most of the time, that text is also correct. But the tone doesn’t change when it isn’t.

A junior colleague who is unsure hesitates. They say “I think”. They look at the floor. AI usually doesn’t. So the signal you have used for your whole career — how confident does this person sound? — stops working.

You need a different signal. Not distrust of everything, which makes the tool useless. A routine.

Step 1: Decide what a mistake would cost

Before checking anything, ask one question: if this is wrong, what happens?

• Brainstorming titles for a presentation? A wrong answer costs nothing. Use it freely.
• Summarising a document you have in front of you? Low cost, if you spot-check it against the source.
• A figure going into a client proposal, a clause in a contract, a technical setting in production, a medical or legal question? High cost. Never use it unchecked.

The level of checking should match the cost of the error, not your level of trust in the tool.

Step 2: Separate the claims from the prose

A long answer contains a handful of claims that matter and a lot of connective text. Pull out the claims: names, numbers, dates, rules, quotes, references, technical commands.

Those are the parts you check. The rest is packaging.

Step 3: Check the claims against something that isn’t the AI

Asking the same assistant “are you sure?” is not verification. It may apologise and change a correct answer, or confirm a wrong one.

Check against an independent source:

• A number → the original report, filing or dataset, not an article quoting it.
• A law or rule → the official text, and its current version.
• A reference or paper → search for it. If you can’t find it, treat it as invented until proven otherwise.
• A quote → the original speech, interview or document.
• A piece of code → the current official documentation, and a test run.

If the AI gives you a link, open it. Check that the page exists and that it actually says what the answer claims.

Step 4: Watch for the classic weak spots

Some areas fail more often than others. Slow down when the answer involves:

• Recent events. Every model has a knowledge cutoff. Anything that may have changed since then — prices, roles, regulations, product versions — needs a fresh check.
• Precise numbers. Percentages and statistics with no source are a red flag.
• Niche topics. The less written about something, the more the model fills gaps.
• Things you asked it to agree with. A leading question (“Isn’t it true that…”) tends to get a leading answer.

Step 5: Make the tool show its uncertainty

You can change how the conversation works. Ask the assistant to:

• mark which statements it is unsure about;
• separate facts from its own reasoning;
• say “I don’t know” instead of guessing;
• list what you should verify before using the answer.

This doesn’t make the output true. It makes the weak points visible, which is what you need.

The point is not to trust less. It’s to trust on purpose.

In critical infrastructure we don’t ask whether a component is trustworthy in general. We ask under which conditions, for which function, with which checks. The same discipline applies here.

AI is an excellent first draft and a poor final authority. Keep the authority for yourself.

Your first step

Take the last AI answer you used at work. Underline every number, name, date and rule in it. Verify the two that would hurt most if they were wrong. That’s the whole routine — now it’s a habit.

Back to Surprise Me — choose another fear to name.