No Stake in the Game
A reader wrote in about the Magnifica Humanitas piece — admitting he knew nothing of the encyclical, the event, or Chris Olah. What began as a note on borrowed legitimacy turned into something harder: not whether AI understands, but who bears the cost when it decides.
Since you consistently invited readers to engage in the conversations started with your blog posts, here is my take on Your recent blog on Magnifica Humanitas and Chris Olah's participation in its unveiling.
I should probably start by admitting that until I read your piece, I knew bugger all about either the encyclical or the event itself. And I’d never heard of Chris Olah.
Vatican announcements don’t compete for my time, nor am I a catholic. But thanks for bringing this to my (and other followers) attention.
The appearance of Chris Olah alongside Pope Leo didn’t initially strike me as particularly remarkable. Actually reading your blog I was immediately reminded of Godfather III when Corleone met the Cardinal who soon after became Pope.
People borrow legitimacy from one another all the time.
Politicians do it for a living. Universities do it. Corporations do it. Most of us probably do it occasionally. Put two prominent people on the same stage and each acquires a little reflected credibility from the other. Well they think they do. Think Trump and Infantino after the World Cup final!
At first glance, there didn't seem much more to it than that.
And yet your article has lingered with me.
Not the event itself so much as the questions it raised.
What surprised me was where those questions eventually led.
One of the themes in your essay that particularly resonated with me was the nature of uncertainty.
What I found interesting was not so much the Vatican's involvement with AI, but the question of what AI actually lacks.
Like many people, I initially accepted the Pope's claim that AI does not truly understand what it produces.
The more I have reflected on that statement, however, the less certain I have become. And I propose it’s too simplistic.
Over the past year I’ve spent a good deal of time 'talking' with AI systems. I support a significantly sized computer system, the code, and the data. And as I've learnt to work with it my experience has only been positive. It can follow arguments, identify contradictions, revisit earlier points, and engage in discussions that many humans would have abandoned several coffees ago.
This made me wonder whether ‘understanding’ is really the right place to draw a boundary.
After several long conversations on the subject, I found myself landing somewhere quite different.
What seems absent is not necessarily understanding.
AI has no stake in the game.
If I spend several days reflecting on an article such as yours, the article becomes part of my life.
I revisit and reread it; I consider and ponder it.
I even imagine a debate about points with you in my mind.
I might disagree with it or at least disagree with specific statements, if not the tenet.
And of course I connect it to unrelated experiences.
I change my mind about parts of it.
And at the same time I also worry that you, the crew and boat are all safe and so periodically and reassuredly check your track. Isn’t that amazing technology?
In short, as a person, the conversation continues after the conversation has ended and there is concern for welfare and consequence.
The AI does nothing of the sort.
When the exchange is finished, it does not continue thinking.
It does not revisit the discussion while metaphorically staring out of a window.
Most importantly, it has no stake in the outcome.
If I disagree with it, nothing matters to it.
If I disappear, it will not wonder where I have gone. It will not give me a call to check in like my brother does.
If I never return, it will not miss me.
It is in the moment, it exists only in the moment.
This struck me as potentially more significant than whether AI understands in precisely the same way humans do.
Human beings participate in the consequences of their decisions.
AI participates in the decisions but not the consequences.
Take autonomous vehicles as an analogy.
The debate is usually framed in terms of capability: can the AI drive more safely than a human?
That is certainly an important question.
Yet the deeper question may be: who bears the consequences when things go wrong?
The family does.
The manufacturer does.
The insurer does.
Society does.
The AI does not.
The future matters to us because we must live in it.
An AI agent has no equivalent stake.
Oddly enough, this line of thought led me somewhere entirely unexpected: Bernard of Chartres and the famous metaphor of dwarfs standing on the shoulders of giants. And in checking against my memory I was surprised that the phrase predated Newton by centuries. But that’s irrelevant. Back to the thought!
What struck me was not merely the intellectual content of the metaphor but its human content. (I am debating now if I should use context rather that content, I find those two words a difficult choice, anyway.)
Every one of us knows something about shoulders.
We know what it feels like to carry weight (responsibility).
We know what it feels like to be carried.
We know how easy it is to lose balance.
We know the trust involved.
Those physical realities are wrapped inside the metaphor; it’s literal interpretation. But to a person there’s so much more to the metaphor than this. I won’t bore you but I could easily write so much about what the metaphor can mean. Its complexity and nuance, the variety of interpretations. The feelings that it brings, and also alludes to. And it has carried over 8 centuries, and is still as vivid and relevant and meaningful today.
An AI can analyse the metaphor but only within its framework and criteria.
A human being can inhabit it.
That seems to point toward something I suspect we often overlook in discussions about AI.
Many of our deepest ideas are not purely intellectual constructions.
They are rooted in embodied experience, relationships, vulnerability, obligation and memory.
AI can often discuss these things remarkably well.
But it does not live within them.
Which brings me back to Chris Olah and Pope Leo.
Much has been made of technology meeting religion.
I am not sure that is what I see.
What I see instead are two traditions, both standing on very tall giants, attempting to grapple with questions neither can answer alone.
One tradition has spent centuries thinking about intelligence, ethics, meaning, responsibility and human flourishing.
The other has spent recent decades creating machines that compel us to revisit those questions.
Whether either side has the answers seems doubtful.
But perhaps there is value in recognizing that both are participants in a conversation that began long before either Anthropic or Pope Leo appeared on the scene.
Your article sent me off in directions I was not expecting.
So thank you for your challenging posts.
Related Post
Jean Peccoud