The Guardian published an investigation claiming health experts found inaccurate or misleading guidance in some AI Overview responses for medical queries. Google disputes the reporting and says many examples were based on incomplete screenshots.
The Guardian said it tested health-related searches and shared AI Overview responses with charities, medical experts, and patient information groups. Google told The Guardian the “vast majority” of AI Overviews are factual and helpful.
What The Guardian Reported Finding
The Guardian said it tested a range of health queries and asked health organizations to review the AI-generated summaries. Several reviewers said the summaries included misleading or incorrect guidance.
One example involved pancreatic cancer. Anna Jewell, director of support, research and influencing at Pancreatic Cancer UK, said advising patients to avoid high-fat foods was “completely incorrect.” She added that following that guidance “could be really dangerous and jeopardise a person’s chances of being well enough to have treatment.”
The reporting also highlighted mental health queries. Stephen Buckley, head of information at Mind, said some AI summaries for conditions such as psychosis and eating disorders offered “very dangerous advice” and were “incorrect, harmful or could lead people to avoid seeking help.”
The Guardian cited a cancer screening example too. Athena Lamnisos, chief executive of the Eve Appeal cancer charity, said a pap test being listed as a test for vaginal cancer was “completely wrong information.”
Sophie Randall, director of the Patient Information Forum, said the examples showed “Google’s AI Overviews can put inaccurate health information at the top of online searches, presenting a risk to people’s health.”
The Guardian also reported that repeating the same search could produce different AI summaries at different times, pulling from different sources.
Google’s Response
Google disputed both the examples and the conclusions.
A spokesperson told The Guardian that many of the health examples shared were “incomplete screenshots,” but from what the company could assess they linked “to well-known, reputable sources and recommend seeking out expert advice.”
Google told The Guardian the “vast majority” of AI Overviews are “factual and helpful,” and that it “continuously” makes quality improvements. The company also argued that AI Overviews’ accuracy is “on a par” with other Search features, including featured snippets.
Google added that when AI Overviews misinterpret web content or miss context, it will take action under its policies.
The Broader Accuracy Context
This investigation lands in the middle of a debate that’s been running since AI Overviews expanded in 2024.
During the initial rollout, AI Overviews drew attention for bizarre results, including suggestions involving glue on pizza and eating rocks. Google later said it would reduce the scope of queries that trigger AI-written summaries and refine how the feature works.
I covered that launch, and the early accuracy problems quickly became part of the public narrative around AI summaries. The question then was whether the issues were edge cases or something more structural.
More recently, data from Ahrefs suggests medical YMYL queries are more likely than average to trigger AI Overviews. In its analysis of 146 million SERPs, Ahrefs reported that 44.1% of medical YMYL queries triggered an AI Overview. That’s more than double the overall baseline rate in the dataset.
Separate research on medical Q&A in LLMs has pointed to citation-support gaps in AI-generated answers. One evaluation framework, SourceCheckup, found that many responses were not fully supported by the sources they cited, even when systems provided links.
Why This Matters
AI Overviews appear above ranked results. When the topic is health, errors carry more weight.
Publishers have spent years investing in documented medical expertise to meet. This investigation puts the same spotlight on Google’s own summaries when they appear at the top of results.
The Guardian’s reporting also highlights a practical problem. The same query can produce different summaries at different times, making it harder to verify what you saw by running the search again.
Looking Ahead
Google has previously adjusted AI Overviews after viral criticism. Its response to The Guardian indicates it expects AI Overviews to be judged like other Search features, not held to a separate standard.
