KPMG, a global consulting giant, withdrew its major report on 'agentic AI' after nearly 90% of its 45 citations were identified as AI-generated hallucinations. These included false claims about major companies like UBS and the UK's National Health Service. This large-scale inaccuracy reveals a significant lapse in internal review processes for a high-profile publication.
Consulting firms position themselves as AI revolution experts. However, some fail to apply basic quality control to their own AI-generated content. This creates a critical disconnect between advertised expertise and internal operational rigor.
The incident suggests established institutions struggle to manage AI risks effectively. This could lead to a broader crisis of trust in expert advice, demanding greater human oversight in AI-driven processes to ensure factual accuracy and client confidence.
The Extent of AI Hallucinations
- KPMG's AI report contained approximately 40 AI hallucinations out of 45 citations, according to PCMag.
- Companies stated the report's false claims and fabricated case studies appeared AI-generated, according to The Times of India.
- UBS, the UK’s National Health Service, Swiss Federal Railways, and Transport for London confirmed the report's claims about their AI usage were untrue or misleading, according to TechCrunch and The Indian Express.
The volume of fabricated content and direct complaints from major institutions points to a systemic failure in information verification. This likely stemmed from over-reliance on generative AI without sufficient human oversight, compromising the report's accuracy.
