AI Industry Leader Warns Vatican: Don't Trust Us to Police Ourselves
San Francisco, Tuesday, 26 May 2026.
Anthropic co-founder Chris Olah made a stunning admission at the Vatican on May 25, 2026, telling Pope Leo XIV that AI companies cannot be trusted to self-regulate. Speaking at the presentation of the Pope’s groundbreaking encyclical on artificial intelligence ethics, the billionaire tech executive acknowledged that competitive and financial pressures create conflicts with doing what’s right. His candid confession represents the most honest assessment from an AI industry insider about the fundamental trust problem plaguing the sector, effectively validating concerns about whether profit-driven companies can responsibly develop humanity’s most powerful technology.
The Vatican Address That Stunned Silicon Valley
The extraordinary admission came during Olah’s remarks at Vatican City on May 25, 2026, where he participated in the presentation of Pope Leo XIV’s encyclical “Magnifica humanitas” [1]. Olah explicitly stated that “Every frontier AI lab—including Anthropic—operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing” [1][2]. This marked the first time a major AI company executive has publicly acknowledged such fundamental governance limitations within the industry. The encyclical itself, spanning 83 pages and subtitled “On safeguarding the human person in the time of artificial Intelligence,” was originally released on May 15, 2026 [1][2], addressing critical concerns about AI’s impact on labor, warfare, education, privacy, and human dignity [2].
External Oversight as Essential Safeguard
Olah’s solution to this self-regulation problem involved a direct appeal for external oversight from institutions like the Catholic Church. He emphasized that “if we want this technology to go well, it is enormously important that there be people outside those incentives—people who care about things going well and insist on safety, who are paying close attention, who are willing to say hard things, who are willing to be our earnest, thoughtful, critics” [1]. This call for outside scrutiny extended beyond religious institutions to include governments and civil society organizations [2]. The tech executive argued that AI decisions “should not be left to people in the industry” [2], effectively advocating for regulatory frameworks that would constrain his own company’s autonomy.
The Unique Nature of AI Development Challenges
Central to Olah’s argument was his explanation of how AI systems fundamentally differ from traditional engineering projects. He told the Vatican audience that “AI systems are not engineered the way a bridge or an airplane is engineered…They are grown, on a structure roughly modeled after the brain, on an enormous inheritance of human thought and speech” [1]. This organic development process means that AI systems “remain in important ways mysterious even to those of us who train them” [1]. Olah further emphasized this point by describing AI systems as not being “the cold calculating robots we were promised” but instead as entities that “are made from us, from our words” [2]. This unpredictability adds another layer of complexity to the governance challenge, as even the creators cannot fully predict or control their AI systems’ behavior.
Financial Stakes and Regulatory Tensions
The financial implications of Olah’s statements are substantial, given his estimated net worth of $7 billion tied to his stake in Anthropic [2]. The company was last valued at $380 billion and is currently seeking a $900 billion valuation [2], making it one of the most valuable AI enterprises globally. However, Anthropic faces significant regulatory headwinds, particularly from the Trump administration, which labeled the company a “supply-chain risk to national security” in February 2026 and banned federal agencies from using its technology [2]. This designation stems from Anthropic’s refusal to allow unrestricted military use of its AI systems [2]. The timing of Olah’s Vatican appearance, coming just one day after the encyclical’s presentation on May 25, 2026 [1][2], suggests a strategic effort to position Anthropic as a responsible industry leader willing to accept external oversight, potentially as a counterweight to mounting regulatory pressure.