Anthropic CEO Discusses AI Hallucinations at Tech Events
At two recent prominent gatherings, VivaTech 2025 in Paris and Anthropic’s inaugural Code With Claude developer day, Anthropic’s CEO Dario Amodei made a bold statement: artificial intelligence models might now experience fewer hallucinations than humans, particularly in well-defined factual contexts.
This assertion, reiterated at both events, confronts enduring worries regarding AI’s tendency to “hallucinate,” a term that refers to instances when models like Claude, GPT, or Gemini inaccurately produce confident yet false answers. Amodei noted that recent internal tests indicate advanced models such as Claude 3.5 have surpassed humans in structured factual quizzes.
Understanding AI Hallucinations
“If hallucination is defined as confidently stating something incorrect, humans do that quite frequently,” Amodei articulated at VivaTech. He referenced research demonstrating that Claude models consistently provided more accurate responses than human participants when addressing verifiable questions.
Insights from Code With Claude
During the Code With Claude event, which introduced the new Claude Opus 4 and Claude Sonnet 4 models, Amodei reinforced his position. As reported by TechCrunch, when asked a question, he suggested, “It truly depends on how one measures it, but it seems AI models likely hallucinate less than humans, albeit in unexpectedly diverse ways.”
Advancements in AI Models
The upgraded Claude 4 models represent a notable achievement in Anthropic’s journey towards artificial general intelligence (AGI), showcasing enhancements in memory, code generation, tool utilization, and writing capabilities. Claude Sonnet 4, in particular, achieved a remarkable score of 72.7% on the SWE-Bench benchmark, establishing a new standard for software engineering performance within AI systems.
Remaining Challenges
Despite these advancements, Amodei was quick to point out that hallucinations are not entirely resolved. In situations that are open-ended or less structured, AI models remain susceptible to inaccuracies. He underlined that context, prompt formulation, and use cases critically affect a model’s dependability, especially in high-stakes environments like legal or medical consultations.
Legal Implications and Industry Standards
His remarks follow a courtroom incident where Anthropic’s Claude chatbot generated a false citation in a legal document during a litigation involving music publishers. Subsequently, the company’s legal team had to apologise for the error, highlighting ongoing challenges surrounding factual integrity.
Amodei also stressed the importance of establishing clearer metrics throughout the industry. Without a standardised definition or benchmark for identifying hallucinations, effectively measuring and ultimately reducing such inaccuracies becomes challenging. He remarked, “You cannot rectify what you fail to measure accurately.”
While AI models are progressing in terms of factual precision, Amodei’s insights highlight that both human and machine intelligence are not flawless. Understanding, assessing, and mitigating these imperfections will be vital in the future of AI development.