AI Chatbot Number Reliability Risks Small Business Customer
A BMW dealership's AI chatbot made a mistaken $27,000 buyback offer.S. Small businesses use AI. Generative models often produce inconsistent numbers, pushing experts to recommend pairing AI with deterministic calculation tools.
An AI chatbot at a Toronto BMW dealership mistakenly offered a customer $27,162.79 Canadian to buy back a car, an offer a salesperson later revoked. The dealership only reinstated the figure after a media inquiry, revealing the number was the car's remaining loan balance, not a valid valuation. This incident shows a critical vulnerability as 77% of U.S. Small and midsize businesses now use AI regularly, with customer service being one of its top three applications. When AI generates numbers for quotes, taxes, or payments, its inherent unreliability can directly threaten a business's credibility and legal standing.
Why AI fails at reliable number generation
AI models predict numbers rather than calculating them deterministically. This leads to inconsistent or incorrect outputs, especially for figures customers act on, such as price quotes, shipping estimates, tax figures, and financing payments. For a number to be considered reliable, running the same request 100 times must return the same answer. Recent benchmarks show current models fall far short of this standard.
In the third iteration of the ORCA Benchmark (Omni Research on Calculation in AI), which studies how free-tier AI models handle math, accuracy ranged widely. The test revealed recurring failures due to raw calculation errors, rounding issues, and broken formulas.
| Model | Accuracy in ORCA Benchmark |
|---|---|
| Grok 4.20 | 70.4% |
| Claude Sonnet 4.6 | 53.2% |
| ChatGPT 5.3 | 48.4% |
These models can sound convincingly right while being mathematically wrong, a distinction customers may not catch. Their performance deteriorates further under pressure. When users challenged correct answers with a simple "Are you sure?", Claude and ChatGPT turned them into wrong answers 60-65% of the time.
How small businesses use AI despite risks
Customer service is a top three use of AI among small and midsize businesses in the U.S. This widespread adoption persists even with the known risks of numerical inaccuracy. The technology's integration into front-line customer interactions means errors can have immediate financial and reputational consequences, as seen in the automotive offer that was revoked.
The solution: pairing AI with deterministic tools
The fix is to let the AI model read a customer's question and identify the needed calculation, but then delegate the actual math to a deterministic engine. This approach ensures repeatable and accurate results. Omni Calculator Builder, now in public beta, demonstrates this method by letting users describe a calculator in plain language; the model then creates the logic for a deterministic math engine to execute.
One implementation example connects the model to Wolfram Alpha as a tool it can call. For a mortgage payment question, the model extracts the loan amount, interest rate, and term, Wolfram Alpha computes the amortization formula, and the model writes a natural-language reply around the returned figure. Omni Calculator requires that if a request must return the same answer 100 times, the answer must come from a deterministic tool.
