The cost of LLM hallucinations is the expected cost of wrong but confident AI output: the error rate for a use case, multiplied by how often it runs, multiplied by what each error costs when it reaches a person or a process. That cost includes rework and review time, refunds and goodwill, regulatory and legal exposure, and reputational damage. A useful way to manage it is an error budget per use case: decide the acceptable error rate based on impact, measure the real rate on evaluation sets and in production, and invest in controls such as grounding, citations, abstention, output validation and human review only where they reduce expected cost more than they add.
Key takeaways
- Hallucination cost depends far more on where the output goes than on the model.
- Estimate expected cost per use case: error rate × volume × cost per error.
- Set an error budget per use case and measure against it continuously.
- Grounding, citations and abstention are usually the cheapest controls.
- Reserve human review for outputs where an error is expensive.
Where LLM hallucination costs come from
| Cost type | Example |
|---|---|
| Rework and review | Staff correcting AI drafts, rechecking facts |
| Customer remediation | Refunds, credits or goodwill after wrong information |
| Operational errors | Wrong data entered into systems by an agent |
| Legal and regulatory | Liability for statements made by an AI assistant; regulatory breaches |
| Reputation | Loss of trust after publicised mistakes |
Courts have held companies responsible for what their chatbots tell customers; in Moffatt v. Air Canada, a tribunal ordered an airline to honour information its website chatbot had given incorrectly.
Estimating the expected cost of hallucinations
Expected monthly cost = error rate × monthly volume × average cost per error that reaches impact
“Reaches impact” matters: an error caught by a reviewer costs review time; an error that reaches a customer or a system costs much more. Model both paths. For RAG systems, see how to measure faithfulness in how to evaluate RAG systems.
Setting error budgets by use case
An error budget is the maximum error rate you will accept for a use case before you add controls or stop. It should reflect impact: a brainstorming assistant can tolerate far more errors than a tool that answers customers about refunds or drafts clinical notes.
Controls and their cost
| Control | What it does | Relative cost |
|---|---|---|
| Grounding (RAG) | Answers from approved sources | Medium to build, low to run |
| Citations | Shows sources so users can verify | Low |
| Abstention | Says “I don’t know” or escalates when evidence is weak | Low |
| Output validation | Checks formats, values and policies before use | Low to medium |
| Guardrails | Blocks unsafe or out-of-scope responses; see LLM guardrails | Low to medium |
| Human review | A person checks before output is used | High per item |
| Evaluation and monitoring | Measures error rates before and after release | Medium, ongoing |
A worked example
Illustrative, fictional example: a customer support assistant handling 30,000 questions a month.
| Scenario | Error rate reaching customers | Average cost per error | Expected monthly cost |
|---|---|---|---|
| Ungrounded assistant | 3% | $25 | $22,500 |
| Grounded with citations and abstention | 0.6% | $25 | $4,500 |
| Plus human review of refund answers only | 0.2% | $25 | $1,500 plus review cost |
If grounding costs $40,000 to build and saves $18,000 a month, it pays back in under three months. Reviewing every answer would cost far more than the remaining errors.
Measuring hallucinations
- Build evaluation sets of real questions with verified answers; see how to test LLM applications.
- Score groundedness: is every claim supported by retrieved sources?
- Sample production conversations for human review each week.
- Track user corrections, escalations and complaints.
- Re-run evaluations on every model, prompt or data change; see LLMOps architecture.
Governance for these decisions is covered in AI governance for small businesses.
Common questions
How much do LLM hallucinations cost?
It depends on the use case: error rate times volume times the cost of each error that reaches a customer, process or decision. Internal brainstorming costs little; customer-facing or regulated uses can cost a lot.
Can hallucinations be eliminated?
Not completely, but grounding, citations, abstention, validation and targeted human review can reduce them to an acceptable level for most business uses.
What is an AI error budget?
The maximum error rate you accept for a use case, based on its impact, used to decide when to add controls or limit the use.
Is a bigger model less likely to hallucinate?
Often somewhat, but grounding and controls usually matter more than model size, and larger models cost more per request.
Who is liable for chatbot errors?
Generally the organisation deploying the chatbot. Treat AI statements to customers as statements from your business.
Reduce hallucination risk where it costs most
Share your LLM use cases and volumes. Our generative AI development team will estimate error costs, set error budgets and implement the controls that pay back.