AI Hallucinations Prompt Customers to Overlook Allergies, Raising Health Concerns

In a New York City restaurant, server Madison reports a growing problem: diners who claim shellfish allergies are being served dishes containing shellfish broth after the server’s allergy check. Customers, citing ChatGPT’s contradictory statements, dismiss the server’s warnings. Madison says, “They just want to talk to ChatGPT. You’re allergic to shellfish, and ChatGPT says there’s no shellfish …” The incident highlights how AI hallucinations—misleading or false information generated by language models—can influence consumer decisions in real‑time settings, potentially endangering patrons with severe allergies. The case underscores the need for reliable AI outputs, especially when health‑related data is involved.