In recent tests, Bard, Google's AI chatbot, hallucinated 91.4% of the time, failing to retrieve any relevant papers and achieving 0% precision in its responses, according to a study published in Hallucination Rates and Reference Accuracy of ChatGPT and Bard. Critical information generated by such models can be fundamentally misleading, carrying significant risks for users relying on its factual accuracy.
AI is often heralded as a solution to global challenges and a driver of efficiency. However, its current iterations are prone to fundamental errors and biases that can actively harm users and widen societal divides.
Companies and governments are increasingly deploying AI for critical functions, yet without stringent oversight and a fundamental shift in development priorities from fluency to accuracy and fairness, the technology risks eroding trust and deepening existing societal inequities.
Understanding AI's Factual Unreliability
Leading AI models frequently fail to provide accurate or complete information, challenging their perceived reliability. A study found that hallucination rates were 39.6% for GPT-3.5 and 28.6% for GPT-4, while Bard's rate was a striking 91.4%, according to Hallucination Rates and Reference Accuracy of ChatGPT and Bard Precision rates for these models also varied significantly, with GPT-3.5 at 9.4%, GPT-4 at 13.4%, and Bard at 0%.
These figures reveal that despite rapid advancements, even widely adopted AI models frequently generate convincing falsehoods rather than factual data. The technology, instead of augmenting human capabilities with reliable insights, can actively disseminate misinformation, impacting decision-making across various sectors.
Defining AI's Core Flaws: Hallucinations and Biases
AI's errors are not merely minor inaccuracies; they can manifest as outright fabrications or systemic distortions, particularly in critical fields like medicine. For instance, models like ChatGPT can fabricate citations or misinterpret clinical guidelines, as reported by Nature. A hallucination could be as severe as inventing a medical paper, defined by researchers as incorrect information in any two of the title, first author, or year of publication.
