How to reduce AI hallucinations in support replies

AI hallucinations in support replies are invented specifics: a return window you don't offer, a feature that doesn't exist, a delivery date nobody confirmed. They happen when the model has to answer and doesn't have the facts. You reduce them by supplying the facts (policies in context, help articles as a knowledge source, order data via a connector), by telling the model what to do when it doesn't know, and by having an agent check every specific before sending. ReplyRabbit is built around all three.

Where hallucinations come from

A language model completes text. Asked "when will my order arrive?" with no order data, it produces a plausible delivery estimate because that's what a reply to that question looks like. It isn't lying; it's filling a gap. Every hallucination in a support draft points at a gap in what the model was given.

That's the useful framing: each invented fact is a missing input you can supply.

Close the gaps

Policies. Put return windows, shipping times, warranty terms and what you don't do into Company Context in plain sentences. A model with the real policy doesn't need to invent one.

Help articles and docs. On the Pro plan, turn on your FreeScout Knowledge Base and documentation as knowledge sources. Drafts then search those articles by meaning and link the one that answers the question, instead of paraphrasing from memory. Setup is in Connect FreeScout KB.

Order and product data. For "where's my order" and "does this fit my model" tickets, a store connector (Magento, WooCommerce or Shopify, on Pro) lets the draft see what the customer bought. Without it, the draft can only guess.

Past replies. ReplyRabbit reads the whole conversation, including your team's earlier replies, so it doesn't contradict what was already said.

Attachments. On Team, screenshots and PDFs the customer sends are read and used, so the draft addresses the error in the image rather than an imagined one.

Tell the model what to do when it doesn't know

Add a rule to Response Style: "If you don't have the order details or the policy doesn't cover the case, say what you'll check and when you'll reply, instead of guessing." That single line converts most would-be hallucinations into honest placeholders the agent can fill.

Check specifics, not sentences

Reading a draft for fluency catches nothing; hallucinations are fluent. Read it for specifics instead. Every date, amount, timeline, product name, feature and policy statement is either in your records or it isn't.

• Dates and timelines: confirm against the order or ticket. Drafts know today's date, so relative phrases are calculated correctly, but the underlying dates still come from your records.
• Amounts: refund figures come from the order system, never the draft.
• Features and compatibility: check the product page or ask the product team.
• Policy exceptions: if the draft made one, the agent decides whether it's allowed.

On the Team plan, confidence scoring shows how sure the AI was about each draft, which is a good signal for where to look hardest.

Fix the input, not just the draft

When you catch a hallucination, correcting the draft fixes one ticket. Adding the missing fact to Company Context or publishing the missing help article fixes the next hundred. Make that a habit: every caught invention becomes a line in context or an article in the knowledge base.

Setup for the context fields is in How to set up ReplyRabbit; Getting started with ReplyRabbit covers the first draft. If drafts keep inventing the same thing, Troubleshooting ReplyRabbit and the FAQ can help.