The problem isn't that the AI "lies," it's that it fills in gaps
Language models are trained to produce a plausible answer, not to say "I don't know." When a customer asks something your knowledge base doesn't fully cover, an assistant with no controls will fill the gap with whatever "sounds reasonable" — a return policy that doesn't exist, a deadline that isn't yours, a feature the product doesn't have. It's not bad intent from the model: it's its default behavior in the face of incomplete information.
What a critic pass is
A critic pass is a second verification step, separate from generating the answer, where the system audits what it's about to send. The assistant drafts a candidate answer using the knowledge base articles it retrieved, and before sending it, a second check — with access to those same source articles — evaluates whether every claim in that answer is actually backed by the retrieved content. If it finds an unsupported claim, the answer gets corrected or escalated to a human instead of being sent as is.
Why "just give it more context" isn't enough
The naive solution is to think that if the model has more information in the prompt, it won't make things up. That helps, but it doesn't solve the underlying problem: even with the correct context, a model can over-generalize ("the return policy is 30 days" when the article says 30 days only for a specific product) or blend information from two different articles in a way that neither one individually supports what ended up being said. The critic pass doesn't trust that the generation was faithful — it verifies it after the fact.
The cost of not having one
An assistant that makes something up once loses something more expensive than that one conversation: it loses the trust needed to hand it the entire channel. If a customer gets an incorrect answer about a refund and your human team later has to walk it back, the cost isn't just the mistake — it's that from then on someone at your company is going to want to review everything the AI says before it's sent, which kills the whole point of automating.
What it looks like in practice
When the critic pass detects that an answer isn't well supported, there's an order that matters: first, try to regenerate the answer using strictly what's actually in the retrieved articles; if that leaves the answer incomplete, explicitly escalate to a human agent instead of sending a "watered-down" version; and log the case, because a question the system couldn't confidently answer is the most useful signal there is for knowing what's missing from your knowledge base.
It's not magic, it's architecture
No model, no matter how large, guarantees zero hallucinations by design. What you can build is a system where the responsibility for not making things up doesn't rest on a single generation step, but on a process with explicit verification before the answer reaches the customer. That's what makes an AI support assistant reliable for production, not just a demo that works well for the first ten questions.