AI Customer Support Hallucinations: Real Cases, New Rules, and How to Prevent Them

In 2024, a Canadian tribunal ordered Air Canada to honor a refund policy that didn’t exist — because its support chatbot had invented one, and a customer relied on it. That is what an AI customer support hallucination costs when it reaches a real person: not an awkward transcript, but a binding promise your business has to keep.

Most teams evaluating AI support tools treat hallucinations as background noise — annoying, occasional, the price of automation. That framing is wrong, and it’s getting more expensive. A hallucination in a support context is a defect that creates legal exposure, erodes trust, and, starting in August 2026, can put you on the wrong side of disclosure law. This guide covers what causes them, what they actually cost, and the layered controls that keep an AI agent from making things up — plus how to measure whether yours already does.

What an AI support hallucination actually is

A hallucination is when a model produces an answer that sounds confident and plausible but isn’t grounded in any real source. The model isn’t lying in the human sense. It’s doing exactly what a language model does: predicting the next likely token based on patterns in its training data. When it hasn’t been given the actual answer, it fills the gap with something statistically reasonable — a discount that sounds like your discounts, a policy that resembles policies it has seen, a step that fits the shape of your other steps.

In a chatbot that summarizes a Wikipedia article, that’s a tolerable error. In a support agent that speaks for your company, it’s different in kind. The customer can’t tell a grounded answer from a fabricated one — both arrive in the same confident tone, in the same chat bubble. And in many jurisdictions, a statement your automated agent makes is a statement you made.

The important distinction for buyers: hallucination is not the same as “the agent got it wrong.” There are three failure modes, and they need different fixes.

• Hallucination — the agent answers from training patterns instead of your content, and invents a fact. Fix: grounding and refusal.
• Stale retrieval — the agent answers from your content, but the content is out of date. Fix: re-syncing sources.
• Wrong retrieval — the agent pulls a real but irrelevant source and answers from it. Fix: better retrieval and source hygiene.

Only the first is a true hallucination. The other two are content problems wearing the same costume. A vendor that can’t tell you which one happened on a given conversation can’t fix any of them reliably.

Three patterns that show up in real deployments

1. The invented policy (the Air Canada case)

The most-cited example is Moffatt v. Air Canada. A traveler asked the airline’s chatbot about bereavement fares after a death in the family. The bot told him he could apply for the discounted rate retroactively, within 90 days. That policy did not exist — the real policy required applying before travel. When he requested the refund, Air Canada refused, then argued in front of the British Columbia Civil Resolution Tribunal that it “cannot be held liable for the information provided by the chatbot,” and that the bot was effectively a separate entity.

The tribunal rejected that. It found Air Canada owed the customer a duty of care, that the chatbot’s answer was a negligent misrepresentation, and ordered the airline to pay damages. The ruling came down in February 2024. (McCarthy Tétrault analysis, AI Business)

The lesson isn’t “airlines have bad bots.” It’s that “the AI said it, not us” is not a defense. If your agent can state a policy, it can state a wrong one, and you own the result.

2. The confident discount

A pattern we hear about constantly: the agent offers a promotion, refund, or loyalty discount that nobody created. The customer screenshots it. Sometimes they post it. Either way, support now has to choose between honoring a fabricated offer or telling a customer the AI lied to them. Both are bad mornings. This is a hallucination in its purest form — the model pattern-matched on “companies offer discounts like X” and produced a plausible X.

3. The plausible wrong procedure

The most dangerous category is the one nobody screenshots, because it doesn’t look wrong. The customer asks how to do something — reset a device, cancel before a billing date, configure a setting — and the agent describes a clean, confident, step-by-step procedure that is subtly incorrect. The customer follows it, something breaks or a deadline passes, and the failure surfaces days later as a frustrated ticket that doesn’t even mention the bot. These quietly inflate your ticket volume while looking like deflection wins on the dashboard.

What hallucinations actually cost

Three buckets, in rough order of how often teams underestimate them.

Trust. A single confidently wrong answer does more damage than ten “I’m not sure, let me get a human.” Customers forgive an agent that admits a limit. They don’t forgive one that misled them — especially if it cost them money or time. McKinsey’s 2025 State of AI research found inaccuracy is among the most frequently cited and most frequently experienced negative consequences of generative AI as deployments scale, and one of the top two risks organizations are now actively working to mitigate. (McKinsey, The state of AI) The risk doesn’t shrink as you grow; it compounds.

Legal exposure. Air Canada established the direction of travel: courts and tribunals are treating chatbot statements as company statements. If your agent can make commitments — quote prices, describe policies, confirm eligibility — it can make commitments you are bound to.

Regulation. The EU AI Act’s Article 50 transparency obligations apply from 2 August 2026. Any AI system intended to interact with people must make clear to the user that they are dealing with AI, at or before the start of the conversation. (EU AI Act, Article 50) That rule applies even to organizations with no “high-risk” AI — a customer-facing support chatbot is squarely in scope. Disclosure doesn’t fix hallucinations, but it removes the “the customer thought it was a person” defense and raises the bar on what your agent is allowed to imply. If you sell into the EU, this is a date to put on the calendar now.

A layered framework for preventing hallucinations

There is no single switch. Prevention is a stack of controls, each catching what the one before it missed. Here is the order that matters.

Layer 1 — Ground every answer in your sources

This is the foundation, and it’s non-negotiable. A grounded agent does not answer from the model’s training weights. It retrieves the relevant passages from your knowledge — help center, PDFs, product docs — and is instructed to answer only from those passages. The technique is retrieval-augmented generation (RAG), but the name matters less than the discipline: every answer must trace to a source you control.

Grounding is what turns “what does a refund policy usually look like?” into “what does this company’s refund policy say?” If the source isn’t there, a properly grounded agent has nothing to answer from — which sets up the next layer.

Layer 2 — Make the agent refuse cleanly

Grounding only helps if the agent is allowed to say “I don’t know.” A surprising number of hallucinations come from agents configured to be maximally helpful, which the model interprets as “always produce an answer.” The fix is an explicit instruction: if the retrieved sources don’t contain the answer, say so and offer a next step instead of guessing. A clean refusal that routes to a human is a success, not a failure. An agent that never says “I’m not sure” is not confident — it’s unsupervised.

Layer 3 — Show the citation

Citations are the control that makes the other layers verifiable. When every public answer carries a clickable reference back to the exact source passage it used, three things happen: the customer can check the claim, your team can debug a wrong answer in seconds, and the agent has an implicit constraint — if it can’t cite a source, it shouldn’t be answering. A citation is a contract that says “this came from here.” It turns a black box into something auditable. (How Owlish citations work)

Citations are also your fastest QA signal. A wrong citation tells you retrieval is the problem; a missing citation tells you the agent answered from training data and you need tighter grounding; a correct citation with a wrong answer tells you the source content is wrong or stale. Each diagnosis points at a different fix.

Layer 4 — Hand off before the agent improvises

Some questions should never be answered by automation: account-specific judgment calls, anything touching money or legal commitments, anything the agent has low confidence on. The control here is a handoff that triggers before the model is tempted to fill a gap, and that passes the full conversation context to a human so the customer doesn’t repeat themselves. A handoff is the safety valve for everything the first three layers don’t cover. (Human handoff)

Layer 5 — Pin high-stakes answers

For the handful of questions where a wrong answer is unacceptable — pricing, refund eligibility, security, compliance, cancellation terms — don’t rely on retrieval at all. Author the exact answer and pin it, so those queries return a vetted response verbatim rather than a generated one. In Owlish this is a Direct Response; most serious tools have an equivalent. The principle is universal: the cost of a hallucination on these topics is high enough that you remove the model’s discretion entirely.

Layer 6 — Keep sources clean and current

Retrieval is only as good as what it retrieves. Thin, contradictory, or outdated content produces confident wrong answers even with perfect grounding, because the agent faithfully cites the wrong thing. Re-sync website sources on a schedule, retire stale documents, and resolve contradictions (two pages stating different refund windows will produce inconsistent answers no matter how good the model is). A regular knowledge base gap analysis is part of hallucination prevention, not a separate housekeeping task.

How to measure your hallucination rate

You cannot manage what you don’t measure, and “it seems fine” is not a measurement. A workable QA loop:

1. Sample real conversations weekly. Pull a random set of answered sessions — 30 to 50 is enough to start. Don’t only review the escalations; the dangerous hallucinations are the ones that didn’t escalate.
2. Check each answer against its citation. For every sampled answer, open the cited source and confirm the answer actually follows from it. No citation, or an answer that goes beyond the citation, is a flag.
3. Classify the failures. Sort flags into the three modes from earlier — hallucination, stale retrieval, wrong retrieval. The mix tells you where to invest: grounding, re-syncing, or retrieval tuning.
4. Track the rate over time. Hallucination rate (flagged-as-fabricated ÷ sampled) is the number to watch. A grounded agent on clean content should trend toward low single digits. If it doesn’t, one of the six layers above is missing or misconfigured.
5. Weight by topic. A 2% hallucination rate spread evenly is very different from 2% concentrated in refunds. Always look at the rate on your high-stakes intents, not just the blended average.

This loop is also the evidence you’ll want if a hallucination ever becomes a dispute. “We sample and review weekly, here are the records” is a meaningfully better position than “we assumed the vendor handled it.”

A buyer’s checklist: questions to ask before you sign

If you’re evaluating AI support vendors, these questions separate tools that prevent hallucinations from tools that hope they don’t happen. Ask them on your own content, during a trial, not from the marketing site.

1. Does every public answer carry a citation back to a source? If not, you have no way to verify any answer.
2. What does the agent do when the answer isn’t in the knowledge base? You want “it says it doesn’t know and offers a handoff,” not a confident guess.
3. Can I pin exact answers for high-stakes questions? Pricing and refund policy should not be left to generation.
4. How does handoff work, and does it carry context? A dead-end handoff that makes the customer start over is barely better than a wrong answer.
5. Can I see the hallucination rate on my real questions? The only number that matters is the one the agent produces on your content and your question mix.
6. How do sources stay current? Ask about re-sync scheduling and how quickly an edited document reaches the agent.
7. Does it disclose that it’s an AI? With Article 50 landing in August 2026, this is now a compliance question, not a nicety.

A vendor that answers all seven cleanly is selling prevention. One that deflects to “our model is very accurate” is selling hope.

Where Owlish fits

Owlish is our product, so read this as a vendor being specific rather than a neutral referee.

Owlish is built around the assumption that a confident wrong answer is the worst thing a support agent can do. The agent answers only from the websites, documents, and PDFs you ingest, so replies stay inside content you control instead of the model’s training data. (knowledge base overview) Every public answer can carry a citation back to the exact source passage, which is both a trust signal for the customer and the fastest debugging tool your team has. When the answer isn’t in your knowledge, the agent is configured to say so and hand off to a human with the conversation context attached, rather than improvise. For the handful of questions where a wrong answer is unacceptable, you can pin an exact response with a Direct Response. You deploy it as a web widget or through Slack, Teams, and Discord, and the pricing is flat and session-based — Starter is $49/mo monthly or $39/mo billed annually (save ~20%), with a free tier to test grounding on your own content before you pay anything.

That shape suits small and growing teams that want to automate repetitive questions without betting their reputation on the model’s restraint. It is a weaker fit if you need a full contact-center suite with telephony and CRM-grade ticket routing, or if your support is so judgment-heavy that almost every conversation needs a human anyway — in that case a larger platform or a human-first setup will serve you better, and you should choose one. Owlish is the grounded answering-and-handoff layer, not the entire helpdesk.

FAQ

What is an AI customer support hallucination?

It’s when an AI support agent produces a confident answer that isn’t grounded in any real source — an invented policy, a fabricated discount, or a plausible but incorrect procedure. The model generates it by predicting likely text rather than retrieving a verified fact. It’s distinct from answering from stale or wrong-but-real content, which are content problems rather than fabrications.

Can a business be held liable for what its chatbot says?

Yes. In Moffatt v. Air Canada (British Columbia Civil Resolution Tribunal, February 2024), the airline was ordered to compensate a customer after its chatbot described a refund policy that didn’t exist. The tribunal rejected the argument that the company wasn’t responsible for the bot’s statements. The practical takeaway: treat anything your agent can say as something your company is saying.

How do you stop an AI agent from hallucinating?

You layer controls: ground every answer in your own sources, configure the agent to refuse cleanly when the answer isn’t there, attach citations so answers are verifiable, hand off to a human before the model improvises, pin exact answers for high-stakes topics, and keep your source content clean and current. No single control is enough; the combination is what works.

Does the EU AI Act affect customer support chatbots?

It does. Article 50’s transparency obligations apply from 2 August 2026 and require any AI system that interacts with people to make clear the user is dealing with AI, at or before the start of the conversation. A customer-facing support chatbot is in scope even if you have no other AI. Disclosure doesn’t prevent hallucinations, but it’s now a compliance requirement if you serve EU users.

What is a good hallucination rate for an AI support agent?

There’s no published industry standard, so measure it yourself: sample real conversations, check each answer against its citation, and track the share that were fabricated. A grounded agent on clean content should trend toward low single digits, and you should look at the rate on high-stakes topics like refunds and pricing separately from the blended average, since that’s where a fabrication does the most damage.

Are citations enough to prevent hallucinations?

Citations don’t prevent hallucinations on their own — they make them visible and verifiable. An agent can still cite the wrong source or overstate what a correct source says. Citations work as part of the stack: grounding produces the source, refusal handles the gaps, and citations let you and the customer check the result. Together they turn a black box into something you can audit.

Sources

• EU AI Act — Article 50: Transparency obligations: the disclosure requirements for AI systems that interact with people, applicable from 2 August 2026.
• McCarthy Tétrault — Moffatt v. Air Canada: A Misrepresentation by an AI Chatbot: legal analysis of the tribunal ruling and the duty-of-care finding.
• AI Business — Air Canada Held Responsible for Chatbot’s Hallucinations: summary of the case facts and the company’s failed defense.
• McKinsey — The state of AI: inaccuracy as one of the most frequently cited and experienced generative-AI risks as adoption scales.

Trademark note

Air Canada, Zendesk, Intercom, Slack, Microsoft Teams, Discord, and other names mentioned here are trademarks or registered trademarks of their respective owners. Owlish is not affiliated with or endorsed by those companies unless explicitly stated. Case facts, regulatory dates, and figures were checked against the linked public sources in June 2026.

Where to start with Owlish

If you want an AI support agent that’s honest instead of merely confident, start by grounding one in your real help content and turning on citations, then run the QA loop above on its first week of conversations. Read the knowledge base overview, see the pricing page for plan details, and walk through building your first agent. You’ll know within a few days whether your automation is answering from your sources or making things up.