Pope's AI Adviser Paolo Benanti Warns Big Labs Act Like Cartels, Says Doom Talk Distracts From Governance

Paolo Benanti, the Franciscan friar and ethics professor who serves as the Vatican's point person on artificial intelligence, says the loudest argument in AI right now — whether a godlike machine might wipe out humanity — is pulling attention away from a duller but more urgent question: who writes the rules, and for whose benefit. In remarks to WIRED, Benanti argued that the fixation on existential risk has crowded out public deliberation over how AI should be governed. His concern is less about a rogue superintelligence than about a handful of companies setting the terms of a technology that touches everyone.

Benanti's warning about cartel-like behavior among the largest AI labs lands as regulators in Brussels, Washington and London wrestle with how much concentration to tolerate in the sector. A small group of firms controls the most capable models, the compute needed to train them and much of the talent that builds them, which gives those companies unusual leverage over standards, pricing and access. Benanti's framing suggests the danger is structural rather than theatrical — not a machine turning on its makers, but a market in which a few players can shape the rules that bind their rivals and their users.

The friar is not a technophobe. A member of the Order of Friars Minor and a professor at Rome's Pontifical Gregorian University, he has spent years studying the ethics of emerging technology and helped shape the Vatican's calls for human-centered AI, including its push for a binding international treaty on the technology's development and use. That work has made him a familiar voice in rooms where AI policy gets made, even though he holds no regulatory power himself. His argument is that moral and political choices are already being baked into systems by the companies building them, usually without any public vote.

His criticism echoes a broader complaint from researchers and civil-society groups who say the safety debate has been colonized by the labs themselves. When the discussion is framed around a hypothetical future catastrophe, the companies at the center of it can position themselves as the responsible stewards best equipped to prevent it. That dynamic, critics argue, tends to produce voluntary commitments and self-reported evaluations rather than enforceable rules — and it leaves out the workers, artists and ordinary users already affected by automated decisions.

What Benanti wants is a wider conversation, one that treats AI governance as a matter of ordinary politics rather than a showdown between prophets and skeptics. He has consistently called for international coordination, transparency about how models are trained and tested, and protections for people harmed by automated systems. Whether that kind of debate can gain traction is the open question, especially as governments weigh lighter-touch approaches and the largest labs keep expanding. The next test will be whether lawmakers take up concentration and accountability with the same energy they have given to speculative doom scenarios.