Google AI Gives Wrong Health Tips

Google's AI Overviews have been found to give incorrect and sometimes dangerous health advice. Experts are worried this could hurt people who trust the information. Google is working to fix this problem.
By Newsroom | Technology, Health |

Misleading AI-Generated Health Advice Poses Significant Risk to Public Safety

A recent investigation has brought to light serious concerns regarding the accuracy and safety of health information provided by Google's AI Overviews. Reports indicate that these AI-generated summaries, intended to offer quick answers to user queries, have on numerous occasions dispensed incorrect and potentially harmful medical advice. This situation has triggered widespread alarm among health professionals and patient advocacy groups, who warn that such inaccuracies could lead to delayed diagnoses, improper treatment, and severe health consequences for individuals relying on this information for critical health decisions.

Google puts users at risk by downplaying health disclaimers under AI Overviews - 1

Investigation Uncovers Pervasive Inaccuracies in Health Guidance

An extensive investigation, primarily led by The Guardian, has detailed multiple instances where Google's AI Overviews presented demonstrably false or misleading health information. The findings suggest that the AI summaries have been particularly problematic for searches related to mental health conditions, including eating disorders and psychosis, offering advice that experts have described as "very dangerous."

Read More: New Way to Make Apps Uses Simple Words, But Good Instructions Are Key

Google puts users at risk by downplaying health disclaimers under AI Overviews - 2

  • Specific Examples: The investigation uncovered cases where AI Overviews provided inaccurate guidance, such as suggesting the consumption of non-edible items like rocks or glue, and offering incorrect information about medical tests.

  • Expert Concerns: Health charities and professionals have expressed grave concerns. Lamnisos, an individual quoted in the reports, stated, "If the information they receive is inaccurate or out of context, it can seriously harm their health." Stephen Buckle, head of information at mental health charity Mind, described some AI Overview advice on eating disorders and psychosis as "incorrect, harmful or could lead people to avoid seeking help."

  • Broader Implications: Experts argue that misleading medical information can cause delays in diagnosis and treatment, thereby directly harming patients. The scientific integrity of health information is considered non-negotiable, especially when public health is at stake.

Google's Response and Subsequent Actions

Google has acknowledged some of the issues, stating that it continuously works on improving the quality of its AI Overviews, especially for health-related topics. The company asserts that the vast majority of AI Overviews provide factual and helpful information.

Read More: People Want to Eat Less Processed Food, But It's Hard to Know What That Means

Google puts users at risk by downplaying health disclaimers under AI Overviews - 3

  • Initial Defense: Google initially defended its AI Overviews, claiming they only appear for queries where the quality of the response is deemed high. The company also stated that some overviews link to reputable sources and include advice to seek expert opinions.

  • Removals and Adjustments: Following the revelations, Google has reportedly removed some AI Overviews for specific health-related queries. However, concerns persist that many inaccurate summaries remain active.

  • Ongoing Improvement: A Google spokesperson reiterated the company's significant investment in the quality of AI Overviews, particularly for health topics, and their commitment to ongoing quality improvements.

Methodological Concerns: Source Reliance and Confidence Fluctuations

A significant aspect of the problem lies in how the AI generates its summaries. Studies suggest that AI Overviews may not always distinguish between robust scientific evidence and less reliable sources.

Read More: Hideki Sato, Who Helped Make Sega Game Consoles, Has Died

Google puts users at risk by downplaying health disclaimers under AI Overviews - 4

  • Source Prioritization: A study analyzed health-related prompts and found that Google's AI Overviews cited YouTube more frequently than established medical websites, hospitals, or government health portals. This reliance on potentially less authoritative video content for medical information raises particular alarm.

  • Inconsistent Information: Reports also highlight inconsistencies, with the same health queries sometimes yielding drastically different AI-generated answers at different times. This variability is concerning, especially when compared to stable medical knowledge.

  • Confidence vs. Accuracy: Experts question the AI's self-declared "high confidence" in its responses, as investigations have revealed significant inaccuracies despite such assertions. The AI's ability to accurately assess the validity of its own generated information remains a point of contention.

Expert Opinions and Calls for Safeguards

The medical and AI communities have weighed in, emphasizing the critical need for robust safeguards and ethical considerations.

Read More: Scientist Tests Microwave Device on Himself, Gets Havana Syndrome-Like Symptoms

  • Trust and Authority: The "confident authority" with which AI Overviews present information is seen as particularly risky for health queries, as users may not critically evaluate the information.

  • Regulatory Oversight: There are calls for regulators to collaborate with medical experts and patient representatives to establish proper safeguards for AI-driven search results.

  • AI Limitations: The ongoing issue of AI "hallucinations"—generating fabricated or nonsensical information—continues to plague language model-based tools, underscoring the inherent challenges in their application to sensitive domains like healthcare.

Conclusion and Future Implications

The investigation into Google's AI Overviews has revealed a critical gap between the tool's intended purpose and its real-world impact on user health. The prevalence of misleading and dangerous health advice necessitates immediate and comprehensive action from Google.

  • User Risk: The primary implication is the tangible risk to public health posed by inaccurate AI-generated medical guidance. This can range from minor inconvenconveniences to severe health detriments.

  • Trust Erosion: Repeated inaccuracies could erode public trust in search engines as reliable sources of health information, potentially leading individuals to seek information from less credible avenues.

  • Need for Accountability: The situation underscores the urgent need for greater transparency, rigorous testing, and accountability in the development and deployment of AI technologies, particularly in high-stakes areas such as healthcare. Further investigations are required to ascertain the full extent of the problem and to ensure that robust safeguards are implemented to prevent future occurrences.

Sources Used:

Read More: ByteDance Will Limit AI Video Tool After Copyright Complaints

Frequently Asked Questions

Q: What is the problem with Google's AI Overviews?
Sometimes, the AI gives wrong or even harmful health advice. This can be dangerous for people looking for help.
Q: What kind of wrong advice has the AI given?
The AI has given bad tips, like suggesting people eat things that are not food. It has also given wrong details about medical tests and treatments.
Q: Are experts worried about this?
Yes, doctors and health groups are very worried. They say wrong health advice can cause serious harm and make people avoid getting real help.
Q: What is Google doing about it?
Google knows about the problem and says it is working to make the AI better. They have removed some bad answers and are trying to improve the quality of health information.
Q: Where does the AI get its information?
It seems the AI sometimes uses less trusted sources, like videos, more than official health websites. This can lead to wrong answers.
Q: Is this a big problem for everyone?
Yes, it is a big problem because many people use Google to find health information. Wrong advice can lead to people getting hurt or not getting the right care.