The Sweet Lie of Conscious AI
A question
Let’s begin with a simple question: Are robots persons? The question may sound odd, for given robots as we know them today the answer seems obviously to be ‘no’. But imagine there were robots that not only look human but, thanks to a very sophisticated AI, also behaved like humans. The answer would not be so obvious, then. Apart from the difference in obviousness, however, nothing would have changed. Most importantly, there would still be two possible answers: yes or no. Further, the question would still be an ontological question, one about being (“Are robots persons?”). From this ontological question one must sharply distinguish the epistemological question of how we know whether a robot is a person or not. (In fact, failure to distinguish the two question is perhaps the most common error committed in debates about the nature and limits of AI. It is also the main error – in so far as it is an error, and not a deliberate strategy – of the hereafter critiqued movie The Creator).
We will come back to the two questions and a deeper exploration of them in the last section. First, let’s see how the recent and acclaimed movie The Creator addresses them.
How the film treats the question
The most important, and also most appalling thing to be said here is that the movie does not really address the ontological question at all. What it instead does is this: it present us a world with robots that not only look human, but also behave like humans (the so-called “simulants”); and it creates, through the behavior and choices of the heroes as well as of the villains, an atmosphere that makes the viewer feel as if the simulants (and actually other non-human-like robots as well) were persons.
The story of The Creator plays in the 2060s. There is war between humans (in fact, the United States or the West in general) and AI robots as well as their human supporters. Some AI was responsible for (oh, wait, responsibility is a term pertaining to persons, so let’s rehash this: caused) a nuclear strike on Los Angeles in which one million people died. The declared goal of the human alliance is to eradicate the sophisticated AI bots from the face of the earth (however not all AI, because they themselves use some more “docile” robots as weapons). To this end, they’ve built a super-weapon called Nomad, a battlestation that constantly levitates in the atmosphere and can strike with deadly missiles whenever and wherever it chooses. (There is some reminiscence of the Death Star from Star Wars).
Joshua is an American elite soldier who previously infiltrated the AI movement (whose home is by the way East Asia), but fell in love with the head of the AI community, Maya. Maya is pregnant at the film’s opening. During a Nomad strike, she is severely injured and falls into a deep coma and loses her child (as it turns out later). Joshua loses his right arm in the strike and ceases to be a soldier, working as a cleaning force in the streets of L.A. – until two high-ranking militaries seek him out to look for “Nirmata” (Nepalese for “The Creator”), the human creator of an AI super-weapon that could take down even Nomad.
Joshua indeed finds the super-weapon, a child-like simulant called “Alpha Omega” (Joshua calls it “Alphie”). Of course, the US forces are hunting Alphie, and Joshua, who up until that point has been a stout denier of AI personhood (he keeps telling his colleagues “It’s not real, it’s just a thing”), wondrously develops some affection of Alphie, rescuing it from its hunters. In the end, he even sacrifices himself for Alphie, which was modeled by Maya (who turns out to be Nirmata) after her and Joshua’s unborn child.
In a way, the movie answers the ontological question in a straightforward way: through Joshua’s insisting that despite appearances, the robots aren’t persons, and through Alphie’s declaration that she is no person (and thus cannot go to heaven). But what happens after the first third completely counteracts this clarity on the matter, albeit not with arguments or insights. Instead, the viewer is inundated with sentimental and pseudo-spiritual images that almost sub-consciously suggest that these beings, which behave so human-like, must be persons:
· Drew bemoans his destroyed AI “girlfriend”.
· Maya highlights the peacefulness and harmony of the AI community.
· Joshua defends Alphie like his own child, even dies for it.
· Towards the end, Joshua meets an AI imitation of the by then deceased Maya and runs towards it as if it really was his wife.
· The Americans are clearly depicted as callous, block-headed villains; by the same token, we have a “David vs. Goliath” trope (the AI community is hopelessly outgunned). It is clear where the sympathies are supposed and tend to lie.
But all of this is compatible with the simulants being just highly sophisticated machines that can perfectly imitate human behavior. Whether this imitation is proof of them being persons, or what else would be proof of it, is sidestepped by the movie. It is thus not a helpful contribution to the already confused debate, where influential thinkers like Nick Bostrom suggest that human-like behavior (as for instance found in ChatGPT) is proof of at least some degree of consciousness, which is of course a feature of persons. It is, however, not at all clear that human-like behavior (if it really is human-like) entails the presence of a person, and what value the behavioral evidence has. To these questions we turn in the final section.
A deeper exploration of the question
Now, are AI bots persons (or could they become persons), and how would we know? I’ll address the epistemological question first. It is quite obvious that in order to know personhood, we need some observable data. Those data must of course be behavioral data – first and foremost language, but also facial and bodily reactions. We cannot look “into” a robot, what its thoughts or feelings are (if it has any at all). In fact, we cannot do that even with our fellow humans! Each one of us knows of himself that he is conscious, has understanding and logic, and we rightly assume that of all other human beings as well, but we have no sure proof that we’re right. After all, the other people could be very sophisticated AI robots that perfectly imitate human behavior. This thought experiment is a variation of David Chalmers’ “zombies”, and it seems (at least to me) that such a scenario is conceivable.
But back to our question. Assuming that there could be robots whose appearance and behavior is indistinguishable from those of a human being, would this be proof that those beings are persons? It wouldn’t. If we cannot even know this about our fellow humans with absolute certainty, we cannot know of those robots either. Of course the assumption would be reasonable, at least prima facie. And, lack of such behavioral evidence would be proof that those beings are not persons. The bottom line is that human-like behavioral evidence is a necessary condition for knowledge of personhood, but not a sufficient condition.
As far as knowledge is concerned, we will, for the reason just outlined, by no means get any farther. We must therefore answer the ontological question without recourse on any observable data. Let me for the sake of brevity reframe the question a bit: What reasons do we have to believe that computational machines cannot possibly become persons? In order for something to be a (fully developed and healthy[1]) person, it must have the following features:
· Consciousness, including self-consciousness
· Rationality
· Language
The question whether a robot could ever become a person then becomes the question whether a material object that performs computations can ever develop (self-)consciousness, rationality or language.
And this is precisely where the rub lies. There is no good reason to believe that a computational machine, no matter how complex, could ever develop even one of those features.
Let’s start with consciousness. Phenomenal consciousness – the “what it is like” to see red, to touch velvet, or to hear a bird chirping, and so forth – is so utterly different from any physical state (among which are to have a mass, or a velocity, or angular momentum) that it led it the vast majority of philosophers (historically speaking) to believe that it cannot possibly be physical. More recent attempts at reducing consciousness to physical states have failed; a consequence of which is that a position called property dualism (championed among others by David Chalmers) has become popular. Property dualism admits that there are conscious properties, but not that there are immaterial selves; and that is its main problem. For my consciousness is had by me, a self, and it is a unified consciousness. How a brain, which consists of myriads of parts, is to give rise to both a self and its unified consciousness remains utterly mysterious. But property dualism is the only plausible possibility how consciousness could arise in a machine, for it does not deny that conscious properties arise out of complex physical matter. If property dualism fails, there is no way how a material being can ever become conscious.
Rationality and language can be treated together, because of a common feature that will enter the scene shortly. Before, it is worth pointing out that those two features are what many people believe AI can most easily develop – if it hasn’t already done so. Aren’t chatbots like ChatGPT astounding? Don’t they give reasonable answers even to tricky questions, thereby showing both linguistic capabilities as well as reason? Even if they consistently did, there would still be the question whether the AIs understand what they are doing. Understanding is an essential feature of both human rationality and language. For language it is obvious – we are not machines that produce sounds but actually understanding the meaning of what we’re saying or hearing – for rationality perhaps less so. The rise of computers has presented us with a different sort of “rationality”, one that in many ways is superior to ours: computation. But computation can be had completely without understanding; in computers, it is just a manipulation of symbols according to rules, be they deterministic or probabilistic (neural networks, to which modern chatbots belong, are not fundamentally different here). This is the gist of John Searle’s famous “Chinese Room argument”: Imagine a man who understands only English sits in a room and is handed questions in Chinese. He also has a rulebook at his disposal which assigns each question an answer in Chinese. He can thus hand out sensible answers to the questions in Chinese, but that of course does not mean he understands even a bit what the questions or answers mean. The man is doing the work of a computer; hence, computers do not understand.
Of course, these arguments are but a rough sketch, and may be attacked or even refuted. But this is the kind of discourse we need; not manipulation by sheer cinematographic force.
[1] To require all these features for personhood is a tricky issue, for it may wrongly deny personhood for beings that should clearly be assigned it, like infants, disabled or mentally ill people. This is why I added those qualifiers; to get personhood for people who do not evince at least one of the mentioned features one should understand them as having those features potentially.
Image: Screenshot Disney+