📖 Glossary

AI hallucination: what it is and how business bots avoid it

Updated 2026-09-19 · 4 min read · By the Convo360 team

In short

AI hallucination is when a language model states something false or made-up with full confidence, because it's predicting plausible-sounding text rather than looking up facts. Business bots reduce it by grounding answers in a knowledge base (RAG), writing tight system prompts, adding guided fallbacks for sensitive topics, and testing before launch. It can't be eliminated entirely, so a human hand-off path always matters.

Key takeaways

Hallucination happens because the model predicts likely text, not because it 'chooses' to lie.
A knowledge base with RAG grounds answers in your actual documents instead of the model's general training.
A tight system prompt that says 'only answer from the knowledge base' cuts confident guessing sharply.
Sensitive topics — pricing exceptions, medical, legal, refunds — are safer as guided (no-AI) steps.
Preview / Try-it testing before you publish a flow catches most hallucination risks early.

What AI hallucination actually means

A language model doesn't look things up the way a search engine does. It generates the next most likely word based on patterns learned during training. Most of the time this produces accurate, useful answers, but when the model doesn't actually know something, it can still generate a fluent, confident-sounding answer that is simply wrong.

This is different from a bug or a crash. The bot doesn't fail visibly; it answers normally, with the same tone and confidence as a correct reply, which is exactly why hallucination is risky for a business: the customer often can't tell the difference between a genuinely correct answer and a fabricated one until something goes wrong downstream, like an order that doesn't match what was promised.

The term comes from AI research, but the underlying behaviour is simple: ask a general-purpose model a very specific factual question, your exact stock levels, your exact refund window, a legal clause in your terms, without giving it the actual facts, and it will often fill the gap with something that sounds right instead of saying it doesn't know.

Where it shows up in a business bot

Inventing a product feature, size or price that doesn't exist, especially for a variant that isn't in the knowledge base.
Confidently stating a return or refund policy that's slightly, or completely, wrong.
Answering a medical, legal or financial question with specific advice it isn't qualified to give.
Making up an order status or delivery date when it doesn't actually have access to that data.
Blending two similar but different policies, say, festival-sale terms and regular terms, into one answer that matches neither exactly.

None of these are the bot being 'dishonest'. They're the predictable result of asking a general-purpose model a specific factual question without giving it the facts, or without clear instructions on what to do when it isn't sure.

The fixes that actually work

Retrieval-augmented generation (RAG) is the biggest lever: instead of relying on the model's training data, the bot searches your uploaded documents or URLs first and answers only from what it finds. This is what an AI agent's knowledge base does: upload your policy PDF, product catalogue or FAQ page, and answers are grounded in that content rather than in whatever the model happened to learn during training.

System prompt discipline matters just as much as the knowledge base itself. A prompt that explicitly says 'if the answer isn't in the knowledge base, say you're not sure and offer to connect a human' stops the model from guessing to fill a gap. Vague instructions like 'be helpful and answer customer questions' tend to invite more confident guessing than a prompt that spells out exactly what to do when information is missing.

Lower temperature settings make answers more consistent and less creative, which is usually what you want for a support or sales bot rather than a casual chat companion where some variety in phrasing is welcome.

For anything genuinely sensitive, such as refund exceptions, medical or legal questions, exact contractual terms, a guided (no-AI) step with pre-approved wording is safer than letting the model generate free text at all. See guided vs AI bots for that distinction and when each makes more sense.

Setting this up on Convo360

On Convo360, hallucination controls sit directly in the AI agent builder rather than needing separate tools.

Create an AI agent and upload your actual policy documents, product sheets or website URLs as its knowledge base, rather than relying on a generic system prompt alone.
Write a system prompt that restricts it: answer only from the knowledge base, and if unsure, say so and offer human hand-off instead of guessing.
Keep temperature low for factual bots (support, order status, pricing) and reserve a higher setting only for casual or creative use cases where exact accuracy matters less.
For refunds, medical, legal or contractual exceptions, route to a guided branch or a live-chat hand-off instead of free-form AI Reply.
Before publishing, run Preview / Try-it with the trickiest questions you can think of, ones slightly outside the knowledge base, and check Flow Logs after launch for answers that look off or inconsistent.

Testing and disclosure as ongoing habits

No amount of prompting removes hallucination completely. Treat it as a risk to manage on an ongoing basis, not a bug to fully patch once and forget. Two habits help long after launch: test with real, messy customer phrasing periodically, not just clean sample questions written by your team; and give every AI conversation a visible, easy way to reach a human, so a wrong answer has a quick correction path rather than being the last word the customer hears.

It also helps to periodically re-read a sample of real transcripts in Flow Logs, especially after you update your knowledge base or launch a new product, a knowledge base that's gone slightly stale is one of the most common causes of a previously accurate bot starting to hallucinate again.

Example conversation

Is your Diwali sale return window 30 days or 15?

Let me check that for you — I don't want to guess. One moment... Our Diwali sale items have a 15-day return window, slightly shorter than the usual 30 days. Would you like the link to start a return?

Build a grounded AI agent free for 14 days

Start free in minutes — no code, no credit card. Bring your own AI keys and pay no per-message markup.

Start 14-day free trial →See pricing

Frequently asked questions

Why does an AI chatbot sometimes make up an answer?

Because it generates the most statistically likely next words rather than retrieving verified facts. When it lacks the actual information, it can still produce a fluent but incorrect answer instead of admitting it doesn't know.

What is RAG and how does it reduce hallucination?

RAG (retrieval-augmented generation) means the bot searches your uploaded documents or URLs first and bases its answer on what it finds there, instead of relying only on the model's general training data.

Can hallucination be completely eliminated?

No, not with current AI models. It can be significantly reduced with a good knowledge base, a disciplined system prompt and low temperature, but a human hand-off option should always remain available.

Should I let AI answer medical or legal questions on WhatsApp?

No. The AI should inform generally at most and hand over to a qualified professional for anything personalised — this is standard practice for responsible business bots, not just a Convo360 rule.

How do I test a bot for hallucination before launch?

Use Preview / Try-it with your hardest real customer questions, including ones outside the knowledge base, and confirm the bot says it's unsure rather than inventing an answer.

Does an outdated knowledge base cause more hallucination?

Yes. If your documents describe an old policy or discontinued product, the bot may blend old and new information into an answer that matches neither, so review and refresh the knowledge base regularly.