The Cost of a Hallucination: When AI Becomes a Business Liability

OPINION: For years, we have been told that artificial intelligence is a tool—a productivity booster designed to streamline our lives. But when a software glitch can convince a community that a thriving family business is vanishing, it is time to stop viewing AI as a helpful assistant and start treating it as a liability.

Consider the case of Anna Mae's Bakery and Restaurant in Millbank, Ontario. As reported by CBC News, the business owner, Amanda Herrfort, recently discovered that Google's AI overview feature was telling the public that her establishment would be closing for good on July 31. The cause? A digital hallucination. Google's AI had mistaken Herrfort's bakery for a different business of the same name located in Illinois that actually was closing.

Herrfort described the experience as one of panic, discovering the error via a screenshot sent to her while she was traveling to a wedding. The fallout was immediate and tangible. Despite launching an online campaign to dispel the rumor, Herrfort noted that customers continued to call, email, and walk into the bakery to ask if they were shutting down. Employees were forced to repeatedly tell confused patrons that the business remained open.

Google's response to such errors is a masterclass in corporate deflection. A spokesperson told CBC News that the "vast majority" of these overviews are high quality and that issues arise when there is a lack of high-quality information on the web. The company further claimed that mistakes are simply a byproduct of the billions of searches occurring daily. In the fine print of its summaries, Google admits that "AI responses may include mistakes."

But for a small business owner, a "mistake" isn't a data point to be optimized in the next software update; it is a threat to their livelihood. Christine Logel, a professor of social development studies at the University of Waterloo, told CBC News that such systems were rolled out before testing was complete. Logel warned that when AI is integrated so deeply into the internet, its errors have "real consequences for real people," citing similar hallucinations in the fields of law and science.

This is the danger of the current AI gold rush. Companies are deploying unproven systems into the public square, effectively using small business owners as unpaid beta testers. While Herrfort was eventually able to resolve the issue by asking the AI search bar to fix the error—receiving a response from the AI acknowledging the "detrimental" nature of the mistake—the damage to customer confidence had already begun.

We are told to "buyer beware," as Professor Logel puts it. Herrfort now advises the public to bypass AI summaries and go directly to official websites or Facebook pages to verify information. But the burden of truth should not fall on the victim of the hallucination. When Silicon Valley's pursuit of speed overrides the necessity of accuracy, the result isn't innovation—it's a liability that the rest of us are forced to carry.