If you’ve used ChatGPT, Google Gemini, Grok, Claude, Perplexity or any other generative AI tool, you’ve probably seen them make things up with complete confidence. This is called an AI hallucination — although one research paper suggests we call it BS instead — and it’s an inherent flaw that should give us all pause when using AI.
Hallucinations happen when AI models generate information that looks plausible but is false, misleading or entirely fabricated. It can be as small as a wrong date in an answer, or as big as accusing real people of crimes they’ve never committed.
Don’t miss any of our unbiased tech content and lab-based reviews. Add CNET as a preferred Google source.
And because the answers often sound authoritative, it’s not always easy to spot when a bot has gone off track. Most AI chatbots come with warnings saying they can be wrong and that you should double-check their answers.
Chatbots note that they can make mistakes.
CNET has been covering AI hallucinations since early 2024, when cases began making headlines. A New York lawyer used ChatGPT to draft a legal brief that cited nonexistent cases, leading to sanctions. Google had its fair share of mishaps, too. During its launch demo, Google’s Bard (now called Gemini) once confidently answered a question about the James Webb Space Telescope with incorrect information, wiping billions off Alphabet’s stock value in a single day.
Google’s second fiasco was Gemini’s attempt to show racial diversity, which was part of an effort to correct for the AI bot’s past issues of underrepresentation and stereotyping. The model overcompensated, generating historically inaccurate and offensive images, including one that depicted Black individuals as Nazis.
And who can forget the notorious AI Overviews flop, when it suggested mixing non-toxic glue into pizza sauce to keep the cheese from sliding, or saying eating rocks is good because they are a vital source of minerals and vitamins?
Fast forward to 2025, and similar blunders hit headlines, like ChatGPT advising someone to swap table salt with sodium bromide, landing him in the hospital with a toxic condition known as bromism. You’d expect advanced AI models to hallucinate less. However, as we will see in the more recent examples, we are far from a solution.
What are AI hallucinations, and why do they happen?
Large language models don’t know facts the way people do, and they don’t intend to deceive you. Mike Miller, senior principal product leader for agentic AI at AWS, tells CNET that when data is incomplete, biased or outdated, the system fills in the blanks, sometimes creating information that never existed.
“Hallucinations are inherent to the way that these foundation models work because they’re operating on predictions,” Miller says. “They’re sort of trying to match the statistical probability of their training data.”
Hallucinations can also stem from vague prompts, overconfidence in statistical guesses and gaps in training material. And because most models are trained to respond conversationally, they tend to give polished answers even when they’re wrong in their aim to please you.
A viral X post by Kol Tregaskes, self-described AI news curator, suggests change may be coming, as the GPT-5 model responded with a human-like “I don’t know.” Time will tell whether that was a bug or a new feature that strives to replace hallucinations with honesty (I wish).
GPT-5 says ‘I don’t know’.
Love this, thank you. pic.twitter.com/k6SNFKqZbg— Kol Tregaskes (@koltregaskes) August 18, 2025
Recent large language models hallucinate at rates of 1% to 3%, per AI agent Vectara’s hallucination leaderboard, although some widely used models have a significantly higher rate.
Newer reasoning models, designed to think step by step, surprisingly amplify this issue. Amr Awadallah, co-founder and CEO of Vectara, explains that it takes time to fine-tune these models to reduce the probability of hallucinations, because most reasoning models are new and haven’t been fully optimized yet.
Reasoning models “loop” a bit longer during their “thinking” processes, which gives them more chances to make something up, Awadallah says.
Afraz Jaffri, a senior director analyst at Gartner, agrees that hallucinations remain a problem.
“The newer class of models that are termed ‘reasoning’ models still make mistakes, as the underlying mechanism by which they were trained and the objective they are measured on is still largely the same,” Jaffri tells CNET.
As models are designed to take on more tasks and extend their reasoning, the chance of errors grows. Jaffri explains that one slip early in the process can throw off the entire result.
For example, OpenAI’s report from April this year showed its o3 model hallucinated 33% of the time on person summaries, up from 16% in its o1 predecessor from late 2024. The o4-mini hit 48%. OpenAI has not responded to a request for comment.
(Disclosure: Ziff Davis, CNET’s parent company, in April filed a lawsuit against OpenAI, alleging it infringed Ziff Davis copyrights in training and operating its AI systems.)
How concerning are AI hallucinations?
Sometimes a hallucination is harmless, or even funny. It can be a silly made-up quote, a reading list of non-existent books, a movie release date that’s off by a year, not knowing what year it is or being wrong about a prime number (like in the image below).
Hallucinations in images are probably even more common and a completely different challenge to combat.
But in high-stakes settings, like the fact-driven areas of law and health, the consequences can be serious. The ChatGPT bromism case shows risks because in this case, AI’s unchecked advice caused symptoms of psychosis.
In one other case, Google’s health-care-focused Gemini AI model mistakenly reported a “basilar ganglia infarct,” referring to an imaginary brain part. Fortunately, a doctor caught the error but it raises serious red flags for AI use in medicine. FDA’s Elsa is another example of an AI bot gone rogue. It was supposed to speed up drug approvals but instead hallucinated nonexistent studies.
Then there’s the legal sector. New reports show that AI-generated hallucinations are starting to appear in courtroom filings with fabricated citations, forcing judges to void rulings or sanction attorneys who have over-relied on bad AI output. Damien Charlotin, a Paris-based legal researcher and academic, is actively working to track these legal cases. Others have attempted to record AI hallucinations in academic papers. Another database tracking AI harms has logged more than 3,000 reports.
On top of accuracy issues, AI is even affecting mental health. In a rising number of so-called “AI psychosis” cases — a term describing irrational beliefs about AI’s capabilities — people believe that chatbots are sentient, conspiratorial and all-knowing.
This article from The New York Times shows how AI hallucinations can fuel dangerous delusions. In one case reported, ChatGPT told a man that the world is a simulation, that he should stop taking his prescribed meds and that, if he truly believed, he could fly. Another man, reported to be emotionally attached to an AI character named Juliet, believed OpenAI had “killed” her, leading to paranoia and a fatal police confrontation.
Experts quoted in the piece explain that large language models are designed to validate and extend conversations rather than correct delusions, which can make hallucinations especially harmful by reinforcing conspiracy theories, dangerous behaviors or psychotic thinking.
“The primary goal of model creators is to make chatbots as friendly and helpful as possible, and in doing so, the unintended consequence is that it will give answers or suggestions that are potentially harmful,” Jaffri tells CNET. He explains that hallucinations make this outcome more likely, and the ways it can happen are so vast that even the best safety testing won’t cover every scenario.
Cursor, a developer-focused coding tool, saw its AI-powered customer support bot reportedly hallucinate a company policy that didn’t exist, telling users: “Cursor is designed to work with one device per subscription as a core security feature.” Developers got upset, and the company later clarified this was a hallucination by the AI bot, apologizing and updating its support system to label AI-generated responses.
When AI gives you lemons…
Although it may sound surprising, AI hallucinations aren’t always a drawback. Some people think they can be beneficial and inspire ideas. Hallucinations can potentially fuel creativity in storytelling and art, as fabricating details could spark ideas and help to brainstorm plots.
Here is an example of how to use hallucinations to your advantage:
Most folks are afraid of ChatGPT’s hallucinations: fake citations to papers that don’t exist.
But we can use these hallucinations productively.
I asked ChatGPT to “give me four papers published by Mushtaq Bilal.” ChatGPT gave me four papers, none of which actually exist.
Now I… pic.twitter.com/d8ifXTqa0e— Mushtaq Bilal, PhD (@MushtaqBilalPhD) May 16, 2024
“AI is helpful despite being error-prone if it is faster to verify the output than it is to do the work yourself,” Arvind Narayanan, a director of Princeton Center for Information Technology Policy, also said in a post on X.
Efforts to correct the problem
Tech companies are racing to reduce hallucinations. OpenAI has worked on improving factual accuracy in newer versions of GPT models, while Anthropic says its Claude models were trained with “constitutional AI” to keep outputs safe and grounded. Google has fact-checking layers in Gemini and Perplexity promotes its system of citations as a partial safeguard.
Miller said AWS is working on ways to minimize the risks, including Automated Reasoning checks in Amazon Bedrock, a service for building and scaling generative AI apps, which he says can “help prevent factual errors due to hallucinations with up to 99% accuracy.”
“These checks can also flag ambiguous results, prompting the system to ask users for clarification rather than serving up a wrong answer with confidence,” Miller says.
Experts say fine-tuning models on domain-specific data and prompt engineering can make a big difference in hallucination rates. Prompt engineering is the skill of writing clear, detailed questions or instructions called prompts to get better results from AI.
Approaches like retrieval-augmented generation (RAG) are also being tested, which, instead of relying only on training data, pull in real-time information from trusted sources before answering. That makes them less likely to invent details, although they’re still not perfect.
At the research level, a multi-agent framework checks AI responses across many layers before presenting a refined answer. Another method, still in early testing, claims to eliminate hallucinations entirely by reshaping queries and emphasizing the importance of noun-phrase dominance.
Will AI hallucinations ever go away?
Experts disagree on whether hallucinations can be eliminated. Some say they’re an unavoidable side effect of how large language models work, and that the best we can do is contain the risk through better design and oversight. Others believe that combining AI with structured knowledge bases, real-time fact retrieval and stricter evaluation will eventually drive the rates low enough for safe use in critical industries.
“Hallucination with the current statistical LLM transformer techniques will always be there — they are intrinsic to how the models are built. With that said, I see hallucination rates eventually saturating at around 0.5%,” Awadallah tells CNET.
He clarifies that this estimate applies to “closed” or “grounded” hallucinations, when a model answers using only information drawn from specific documents or data. Rates are higher in “open” settings that draw on all training data, like those that search the internet for an answer.
Always assume potential errors
For everyday users, hallucinations are mostly an inconvenience. If you’re asking a chatbot to draft an email, brainstorm vacation ideas or create a recipe, a wrong fact here or there isn’t catastrophic, especially if you’re double-checking the chatbot’s work. The risk grows when you rely on AI for important factual answers — whether it’s legal, financial, health-related or tied to your identity.
You should treat AI chatbots as assistants, not oracles, and always assume potential error.
“There is also the well-known prompting technique of specifying ‘think step-by-step’ to increase model accuracy,” Jaffri advised. He suggested adding phrases that stress the importance of the task or the cost of making a mistake can help.
Ask for sources, rephrase queries and challenge the bot to self-reflect. Switch between models and warn them about hallucinating. This feedback helps companies improve their AI products. Always double-check important facts, especially when making decisions based on what the AI tells you.
AI hallucinations are one of the defining challenges of this technology. They remind us that, for all their power, chatbots don’t think or understand the way we do. They’re prediction engines, not truth engines.
Until researchers find better safeguards, hallucinations will remain a risk. As Miller says, “If you’re looking for factual information, just take the generative AI responses with a grain of salt.”
Watch this: iPhone Air Is a Wild Card – and Starts a Big Change for Apple
