AI answers with sources: how to stop a support bot from inventing
The dangerous AI answer is not the one that says "I do not know". It is the fluent, well-punctuated paragraph that is wrong about your return policy.
Every support team that tries AI has the same first week. Nine answers out of ten are better than what a tired manager writes at 19:00. The tenth invents a delivery deadline that does not exist, and the customer screenshots it.
That tenth answer is the entire problem, and it is not solved by a better model. It is solved by changing where the answer comes from.
Why a general model gets your policy wrong
A language model on its own answers from a general picture of how businesses usually work. Asked about returns, it produces the most plausible return policy — which is to say, someone else's. It has never read your documents, so it cannot know that you accept returns for 30 days but refund to store credit after 14.
Worse, it has no way to signal that. A model with no information and a model with perfect information produce text with the same confident tone. There is no visible difference between knowledge and guessing.
A support answer is not judged on fluency. It is judged on whether the company will stand behind it.
Retrieval: answering from your own documents
The fix is to stop asking the model what it thinks and start asking it to read. In practice:
- Your documents — policies, FAQs, price lists, product cards, past approved replies — are split into passages and indexed.
- When a question arrives, the system searches that index for the passages that actually address it.
- The model is given the question and those passages, with an instruction to answer only from them.
- The answer comes back with the passages attached, so the manager can see what it was built on.
This is usually called retrieval-augmented generation. The important part is not the acronym: it is that the answer now has a provenance you can check in two seconds.
The source is not decoration. It is the thing that lets a manager approve an answer without re-reading the policy, and the thing that turns "the bot said" into "the document says" when a customer disputes it.
Knowing when it does not know
The second half of the problem is the gap. Sometimes nothing in the knowledge base addresses the question — the customer asks about a product you have not documented, or a situation nobody wrote down.
A system built properly treats "no relevant passage found" as an outcome, not a failure to be papered over. What should happen:
- No confident answer is produced from nothing.
- The question goes to a person, with a note about what was searched.
- The gap is recorded, so the twentieth person asking the same undocumented question is the last one.
That last point is how a knowledge base actually grows: not in a planning meeting, but from the questions that had no answer.
Drafts, autopilot and where the line sits
Two modes are worth separating.
| Draft for approval | Autopilot | |
|---|---|---|
| Who sends | a manager, after reading | the system |
| Good for | anything with money, a complaint or a promise in it | repetitive factual questions with a clear source |
| Risk if wrong | caught before the customer sees it | visible to the customer |
| Where to start | everything, for the first weeks | a narrow list you have watched work |
The honest sequence is: drafts first, for every question, until you have watched a few hundred and know where the model is reliable. Then move a small, specific category — opening hours, delivery cost, stock — to autopilot, and leave everything else to people. Anything touching a refund, a complaint or a legal promise stays with a person permanently.
What to require from a vendor
- Show me the source for this answer. If the interface cannot display which document a sentence came from, nobody can verify anything.
- Show me what happens when the answer is not in the documents. Ask for a live demo with a question the knowledge base cannot cover.
- Tell me what is done with our data. Your documents and conversations should be used to answer your own customers, not to train a model shared with other companies.
- Show me the log. Who approved what, what was edited before sending, which document was used — support work is auditable work.
The measure that matters at the end of the first month is not how many answers the AI wrote. It is how many of them a manager sent without changing a word — and how quickly the questions it could not answer turned into documents it can. See the metrics worth counting and the knowledge base checklist.
Frequently asked
What does it mean that AI answers with a source?
Every generated answer carries the passage from your own documents it was built on. The manager sees the document and section next to the draft and can check it in seconds instead of re-reading the policy.
Can AI answer customers completely on its own?
For a narrow list of repetitive factual questions with a clear source, yes. Anything involving money, complaints or promises should stay a draft a person approves.
Will our documents be used to train a model for other companies?
They should not be. Your documents and conversations are there to answer your own customers; ask any vendor to state this in writing before connecting a knowledge base.