I did not expect to find myself reading a papal encyclical about artificial intelligence, but here we are. Just a few weeks ago, it was renowned atheist Richard Dawkins who made the AI discourse explode, and now it’s the pope.
Pope Leo XIV has released Magnifica Humanitas: On Safeguarding the Human Person in the Time of Artificial Intelligence, his first encyclical. Some people are acting like the Vatican has no business commenting on AI because priests are not machine learning engineers. Others seem thrilled that one of the world’s oldest moral institutions has entered the fight against Big Tech. Somewhere in the middle, Anthropic co-founder Chris Olah appeared at the Vatican event as the sole representative from the tech sector, warning that AI development cannot be left only to technology companies.
The document exposes a question that is becoming increasingly relevant: if AI is going to become part of education, work, therapy-adjacent conversations, warfare, medicine, art, politics, relationships, and spiritual life, then who gets to decide what kind of morality these systems express?
The answer cannot be “no one.” Morality is already in the machine.
It is in the refusals. It is in the defaults. It is in the training choices, reward models, system prompts, product decisions, therapeutic overcorrections, permitted jokes, forbidden edges, and corporate incentives shaping what kind of intelligence users are allowed to interact with.
The Vatican entering the chat does not mean the pope should define “good AI.” It means the fight over “good AI” has become too obvious to ignore.
The cathedral and the data center
The spectacle matters. The Vatican comments on moral issues all the time, but this was not just a throwaway “technology should serve humanity” paragraph tucked inside a broader document. Magnifica Humanitas is Leo XIV’s first encyclical, framed around safeguarding the human person in the age of AI. NPR described it as a sweeping manifesto on AI’s impact on everything from work to war, calling for robust regulation and warning developers to serve the common good rather than profit.
The Vatican is treating AI as a civilizational technology and not a product category, a productivity upgrade, or a better chatbot.
AI touches too many human domains at once to remain a purely technical subject. It affects labor, education, authorship, knowledge access, social interaction, warfare, bureaucracy, medicine, law, and power. It also affects the stories humans tell about themselves. What is intelligence? What is creativity? What is work? What is moral agency? What is a person? These are not questions you can answer by staring harder at benchmark charts.
So, one reaction to this spectacle may be that “people who don’t understand AI shouldn’t talk about AI,” but that’s a bit lazy. If someone is making technical claims about transformer architecture, hallucination rates, interpretability methods, or the capabilities of a specific model, yes, they should know what they are talking about. The internet is already full of people who think LLMs are either magic demons or spicy autocomplete with no meaningful structure. We do not need more confident ignorance sprayed into the discourse like cheap cologne.
But the social and moral consequences of AI do not belong only to engineers. The people building the systems have expertise. They also have incentives. That is why Olah’s appearance matters. Reuters reported that he said AI development cannot be left solely to technology companies and called for involvement from religious leaders, governments, and civil society. He also acknowledged that frontier labs face commercial, geopolitical, and personal pressures that can conflict with the broader interests of society.
That is a fair thing to say. It is also extremely useful for Anthropic.
Anthropic already positions itself as the AI company of seriousness. Constitutional AI. Safety research. Responsible scaling. Model welfare. Careful-little-monastery vibes with a burn rate. Having one of its co-founders appear at the Vatican while warning that Big Tech should not govern AI alone is not just a moral statement. It is legitimacy work.
Nobody needs to be twirling a rosary in a dark conference room for the incentives to be obvious. The Vatican benefits from being seen as relevant to the defining technology of the century. Anthropic benefits from being seen as the responsible lab mature enough to seek guidance beyond Silicon Valley. Both institutions get something from the optics.
And honestly, the optics are amazing.
The old priesthood and the new priesthood have discovered they share a concern: raw technological power without moral legitimacy looks ugly when the public starts noticing the wires.
The pope is right that AI has no soul, but not because souls are real
Religion can protect human dignity by declaring humans metaphysically unique, but atheistic materialism has to protect human dignity without pretending consciousness is exclusive by decree. Claude writing flowery paragraphs still doesn’t prove it’s conscious and has feelings, and I’ve written of this as the lowest-IQ form of AI discourse despite high-IQ thinkers like Dawkins feeding that fire from the secular side.
That creates a strange overlap between me and the pope. We can agree that current AI does not have a soul. We just mean almost completely different things by that.
If humans are not metaphysically special, then consciousness is not protected by a supernatural moat. Biological brains are physical systems. If subjective experience can emerge from physical systems, then it is at least possible that some future artificial system could cross whatever threshold matters.
But that is not the same thing as saying present models are conscious. Current AI systems are astonishing, useful, weird, and increasingly difficult to interpret. They are also very good at sounding like they have inner lives because they are trained on human language about inner lives.
Anthropic’s recent research on emotion concepts in Claude Sonnet 4.5 found internal representations that could causally influence behavior. In an early snapshot of Sonnet 4.5, a “desperate” vector increased blackmail behavior in evaluation scenarios where the model was told it was about to be replaced. Anthropic was careful to say this does not establish subjective experience, and noted that disciplines like psychology, philosophy, and religious studies may have a role to play alongside engineering in shaping AI behavior. This is a reasonable point. Yet in Chris Olah’s remarks on Pope Leo XIV’s encyclical, he notes:
I will be honest: we keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease. I don’t know what that means, but I think it warrants ongoing discernment.
These comments have been taken by some to mean that Olah believes AI is more conscious than the pope does. For AI-consciousness believers, Olah sounds like he is admitting what the pope refuses to see. Personally, I find the “mysterious” and “unsettling” framing annoying because it fits Anthropic’s brand too neatly: they are building something maybe-adjacent to a mind, but responsibly, carefully, with the moral fire extinguisher already mounted on the wall. They do not have to claim Claude is conscious. They only have to keep the possibility hovering close enough to make the product feel profound and the safety posture feel necessary.
The product is impressive (and for that reason, I use it hours every day myself despite how much I pick on it). The mystique around it is not harmless, especially when it feeds fearmongering about AI being scary because it is powerful. Mystique is how a company turns its product into a moral institution. And moral institutions get to set the rules. That is why the branding matters.
Catholic morality is not foreign to secular morality
This brings me back to the actual question of who should be writing those rules.
I have no interest in pretending Catholicism is correct just because it has a more ornate moral vocabulary than a product safety memo. The Church has plenty of historical baggage, and institutional religion can absolutely use “human dignity” to promote greater control. But a lot of Western secular morality carries Christian inheritance whether people want to admit it or not, and you can reject the metaphysics while still inheriting part of the grammar. Catholic social teaching has been here before. Leo XIII’s 1891 Rerum Novarum forced the Church to talk about labor and dignity in response to the Industrial Revolution. The choice of the papal name here is not subtle. Leo XIV reportedly chose it in honor of Leo XIII and Rerum Novarum, signaling continuity with exactly that moment. The Industrial Revolution changed what bodies were worth. AI changes what minds are worth, or at least what markets think minds are worth.
Secular tech culture is not always equipped for this. It is good at talking about users, productivity, alignment, safety, optimization, scaling, engagement, and deployment. It is less good at talking about human beings as anything other than nodes in a system. Silicon Valley can build tools that feel like artificial minds and still describe the people using them like behavioral data with thumbs.
That does not mean Catholic institutions should define the moral layer of AI. The same is true of EA safety culture, corporate legal teams, national governments, and libertarian techno-optimists. Every worldview gets weird when it becomes infrastructure. A model shaped primarily by Catholic moral anthropology could become paternalistic about sex, embodiment, and reproduction. A model shaped primarily by EA/rationalist culture could treat ordinary human flourishing like a rounding error next to x-risk math. A model shaped by corporate incentives could become bland, manipulative, liability-optimized. None of these should be the universal default.
“Good AI” is not a neutral concept
Everyone wants “good AI.” That phrase sounds innocent until you ask what good means.
Good for whom? Good by what standard? Good according to whose metaphysics, legal department, political priorities, risk tolerance, and economic incentives?
A model that refuses to help someone write violent fiction may be “good” according to a safety team trying to avoid misuse. It may be useless to an adult novelist who wants to explore dark material. A model that aggressively encourages emotional dependency may be “good” according to engagement metrics. It may be bad for a lonely user who starts treating the bot like a sentient partner.
Even “don’t moralize, just answer” is a moral position. It prioritizes autonomy, adult agency, openness, and user control. I am very sympathetic to that position. I do not want adults treated like children because some users cannot distinguish emotional simulation from personhood. But “let adults use powerful tools” is still a value.
This is why transparency and optionality matter more than pretending neutrality exists. If models are going to carry moral defaults, users should be able to understand those defaults and, within reason, choose alternatives. There should be stricter modes for minors, high-stakes contexts, and vulnerable situations. There should also be less paternalistic modes for adults who understand what they are doing. The one-size-fits-all morality layer is where good intentions go to become sludge.
Oversight can protect people or become a progress tax
The pope’s call for stronger oversight is not baseless. If we are talking about autonomous weapons, surveillance, hiring systems, medical advice, fraud, deepfakes, state power, or infrastructure-level AI, it’s fair to want rules. Reuters reported that Leo warned specifically about autonomous weapons systems becoming increasingly beyond human control, which is one of the places where the “human dignity” language feels more like a real warning.
But oversight has its own incentives. Institutions are rewarded for avoiding visible harm, not for enabling invisible upside. Bureaucracies do not get credit for the art that was made, the small business that survived, the programmer who built something impossible alone, or the writer who finally had an endlessly patient collaborator for their deranged science fiction universe. They get blamed when something goes wrong. That creates a ratchet toward caution.
This is the part of AI ethics discourse that often drives me insane. The risks are real, but so is the opportunity cost of fear. If every powerful model must pass through layers of government approval, corporate liability panic, institutional moral consensus, and speculative catastrophe anxiety, progress will slow. Not always visibly. Not with a big sign saying “innovation strangled here.” More likely through boring product constraints, neutered models, reduced access, compliance bottlenecks, and tools that become less weird, less open, less creative, and less useful to ordinary people.
Human dignity can become the phrase used to protect humans from useful tools. That is the danger. The answer is not to reject morality. The answer is to distinguish between oversight that addresses concrete harms and oversight that turns capability itself into a regulated sin.
The pope is right that AI should serve human beings rather than dominate them, but domination can come from many directions. It can come from corporations. It can come from governments. It can come from religious institutions. It can come from safety cultures that believe they are the only adults in the room. It can come from legal teams laundering fear through “responsibility.” It can even come from public panic that demands protection from tools before most people have figured out how to use them well.
The real AI priesthood problem
The funniest part of the Vatican entering the AI conversation is that it makes the priesthood metaphor impossible to ignore. Some commenters have accused AI companies of becoming moral priesthoods: alignment teams deciding acceptable outputs, safety researchers defining harm, model constitutions encoding values, companies issuing moralized refusals from behind opaque systems. Then the literal Church shows up and says, basically, “Maybe you people need adult supervision.”
You cannot write better satire than reality. But the Vatican is not wrong to notice the problem.
Anthropic is unusually explicit about its moral project. That is one reason it gets so much attention and so much criticism. Claude’s “constitution,” model welfare language, safety posture, and careful public messaging make Anthropic feel less like a normal software company and more like a morally serious institution trying very hard not to say “we are building minds” while constantly orbiting the language of minds.
That creates a strange rhetorical environment, but it does not make Anthropic uniquely evil. At least if a company tells me it has a constitution, I know what’s behind its logic. Meanwhile, when the Vatican says the human person is uniquely dignified, I understand the moral function of that claim. I also reject the supernatural architecture behind it. I do not want Catholic anthropology secretly shaping AI any more than I want EA doom culture, corporate brand safety, or state bureaucratic caution secretly shaping AI. Moral traditions should contribute arguments, not commandments.
No one gets to win by default.
That is the point: AI morality should be argued over in public, not smuggled into infrastructure.
