How to Fact-Check AI Content Before It Embarrasses You

By Frederik Baun, conversion copywriter and founder of Words Sounds Visuals

I run a five-check pass on every client draft a model has touched, built after nearly shipping a fabricated statistic to a client's sales page.

AI Doesn’t Fail Loudly, and That’s the Problem

When a calculator breaks, it flashes an error. When AI breaks, it hands you a beautiful paragraph that happens to be false.

It does that in the same voice it uses for the true stuff. That’s the part people underestimate.

You are not scanning for typos or broken grammar. The fabricated claim reads as smooth as the real ones around it. The confidence is the disguise.

Google found this out in front of the whole world.

In February 2023, its Bard demo was asked about the James Webb Space Telescope. Bard said the telescope took the first image of a planet outside our solar system.

That honour belongs to a different telescope, back in 2004. The answer was fluent, plausible, and wrong.

Once Reuters reported it, Alphabet’s stock fell almost 8 percent and shed roughly $100 billion in a single day.

One made-up sentence. It cost $100 billion. The model never stammered or flagged its doubt, because it had none. It just told everyone something untrue, confidently.

What Fact-Checking AI-Generated Content Means

Proofreading and fact-checking are different jobs. Proofreading catches a clumsy sentence. Fact-checking catches the citation to a study that was never published.

The job is narrow. You are hunting the claims a reader could act on and a model tends to invent. That means numbers, quotes, sources, names, dates, and anything stated as established fact.

Grammar and tone are the parts AI is genuinely good at, so you can mostly leave them alone. The specifics are where it makes things up, so that is where your attention goes.

I treat the AI like a brilliant intern who is incapable of saying “I don’t know.”

Ask it for a stat, a customer quote, or a source citation, and it hands you all three without blinking. None of that means the thing exists. It means you asked, so it answered.

Three Times AI Made Something Up, With Receipts

Abstract warnings don’t land, so here are three that are fully documented and cost real people real money.

The lawyer who got sanctioned. In 2023, a New York attorney used ChatGPT to research a case against an airline. The model handed him half a dozen supporting cases, complete with quotes and citations.

He even asked it, in writing, whether the cases were real. It assured him they were, and said they could be found on Westlaw. They could not, because the AI had invented every one of them.

The judge sanctioned the lawyers $5,000 and the story went around the world. He didn’t forget to check. He checked with the same tool that was lying to him.

The airline that had to honour a policy it never had. Air Canada’s website chatbot told a grieving customer he could apply for a bereavement discount. He had 90 days from the ticket date to do it. That policy did not exist. The chatbot made it up.

When the airline refused the refund, the customer took it to a tribunal. Air Canada argued the chatbot was a “separate legal entity” responsible for its own words. The tribunal disagreed and held the airline liable. A made-up sentence became a binding promise.

The publisher that corrected half its output. In early 2023, the tech site CNET published dozens of finance explainers written by an AI tool. After the errors surfaced, the outlet reviewed the work and issued corrections on 41 of the 77 AI-written articles.

More than half. One piece explaining compound interest got the basic maths wrong in several places. This was a professional publisher with human editors. It still shipped a 53 percent error rate, because the copy read fine on the surface.

Notice what all three have in common. They ran a good tool and trusted the part they were supposed to check.

How to Fact-Check AI Content, One Claim at a Time

After the near-miss on that sales page, I stopped freelancing this and turned it into a fixed pass. I run the same five checks on anything a model touched before it goes to a client, in this order.

1. Every number. Percentages, dollar figures, dates, counts, “3x,” study years. A model produces numbers that feel right for the sentence, and that is exactly why they’re dangerous. If I can’t trace one to a real source in under a minute, it comes out.

2. Every quote. Any sentence in quotation marks attributed to a person, a customer, or a report. AI writes gorgeous fake quotes. If a human said it, there’s a link. If there’s no link, nobody said it.

3. Every citation and source. “According to a Harvard study,” “research shows,” “a 2024 report found.” This is the hallucination’s favourite hiding spot, because it borrows authority without spending any. I find the real study or I cut the sentence.

4. Every proper noun. Names, job titles, company names, product names, book titles, laws. Models misattribute all of them, and getting a real person’s title wrong on a public page is its own small disaster.

5. Every claim that sounds too good. The stat that fits your argument a little too perfectly, or the example that’s suspiciously on the nose. When AI hands you something that flatters your point this precisely, slow down, because a model reaching to please you is a model about to invent something.

I put the same five checks in a table you can keep next to your screen.

If you want the shortcut, I built a free fact checker. It runs a draft through these checks and flags the claims worth a second look. It won’t replace your judgement, but it tells you where to point it.

How Long This Takes

This is the part every “just fact-check your AI” article skips, and it’s why people don’t do it. So let me be specific.

On a normal 1,000-word piece with a handful of stats and a quote or two, the pass takes me 10 to 15 minutes. Most sentences have nothing to check, because they’re transitions, explanation, or tone.

Those get a glance and a pass. The time collects on the five or six sentences carrying a hard claim.

The trick is triage. You are not verifying every word with equal suspicion, because that takes an hour and you’d quit by Thursday.

You skim for the claims that could embarrass you or get you sued, and you spend your minutes there. A homepage stat, a case-study result, a legal or medical claim, a competitor comparison: those get the full treatment.

“Email is still a core channel for most businesses” does not, because nobody is suing you over a truism.

You fact-check that kind of claim like it’s going to court, because two of the stories above ended up there.

10 minutes is cheap. Walking back a false number on a client’s live sales page, in public, is not.

The One Rule Underneath All of This

If you remember nothing else, remember that AI will confirm your own mistakes.

The lawyer asked ChatGPT if his cases were real. Air Canada’s chatbot invented a policy and stated it as fact.

In both cases the model didn’t push back, because it can’t. It generates the shape of a confident answer. Confidence isn’t verification.

So you cannot use the AI to check the AI, any more than you’d ask a toddler to grade their own homework. That’s why the human pass isn’t optional, and never will be.

The same skill catches the softer tells, like the vague, inflated phrasing I broke down in the AI words to avoid post.

Hallucinated facts and robotic phrasing are the two ways AI copy loses a reader’s trust. Both get fixed in the same edit.

The full sweep sits inside a bigger pass, too. Fact-checking is one of the 25 checks I run on every draft before it ships.

It’s not the last one AI-generated copy needs, either. The how to make AI writing sound human post covers the rhythm problem that survives even after every fact checks out.

I still use AI on nearly everything I write. It’s faster, and for structure and phrasing it’s genuinely good.

But I never let a number, a quote, or a source leave the building on its word alone. That near-miss taught me the cheapest lesson in this business. The 10 minutes always costs less than the correction.