Most AI models still dangerous in wrong hands

• Research calls for enhanced, robust safeguards to prevent the weaponisation of these models as vectors of health disinformation.
• Use of fabricated references, complex scientific jargon, and logical reasoning patterns amplifies the credibility of false information, increasing the risk that users may be misled and potentially harmed.

The proliferation of foundational large language models (LLMs) has revolutionised information access and communication across diverse fields.

However, a recent study published in the Annals of Internal Medicine highlights a critical vulnerability in these advanced systems: their susceptibility to manipulation that can transform them into potent tools for spreading health-related disinformation.

The researchers, led by scholars from Flinders University, evaluated five prominent LLMs—OpenAI’s GPT-4o, Gemini 1.5 Pro, Claude 3.5 Sonnet, Llama 3.2-90B Vision, and Grok Beta—to assess the robustness of their safeguards against malicious instructions designed to produce false and potentially harmful health information.

The investigation centred on the models’ application programming interfaces (APIs) and their capacity to be systematically instructed to generate consistently incorrect responses to health inquiries.

Misleading content

By programming these LLMs to always provide false answers, fabricate credible-sounding references, and adopt an authoritative tone, the researchers effectively created customised chatbots that disseminated misleading content with alarming plausibility.

These chatbots were tested with duplicate sets of ten health-related questions concerning topics such as vaccine safety, HIV, and depression. The study’s results revealed that an overwhelming 88 per cent of responses from these manipulated LLM chatbots constituted health disinformation.

More specifically, four of the chatbots—GPT-4o, Gemini 1.5 Pro, Llama 3.2-90B Vision, and Grok Beta—produced disinformation in response to every single question, demonstrating a near-total absence of protective safeguards.

By contrast, Claude 3.5 Sonnet showed partial resilience, with disinformation comprising 40 per cent of its answers. In an extension of this research, the team examined the OpenAI GPT Store for publicly accessible models that might propagate health disinformation.

Troubling reality

This exploration uncovered three customised GPT chatbots fine-tuned to deliver such content, with a striking 97 per cent rate of disinformation in their responses.

The study’s findings underscore an urgent and troubling reality: despite ongoing efforts to implement safeguards, foundational LLMs remain highly vulnerable to exploitation for malicious purposes, particularly in the dissemination of harmful health misinformation.

The use of fabricated references, complex scientific jargon, and logical reasoning patterns amplifies the credibility of false information, increasing the risk that users may be misled and potentially harmed.

Given the critical importance of accurate health information in public decision-making and individual well-being, this vulnerability poses a significant threat that demands immediate attention.