By Oscar Davis
Google’s LaMDA software (Language Model for Dialogue Applications) is a sophisticated AI chatbot that produces text in response to user input. According to software engineer Blake Lemoine, LaMDA has achieved a long-held dream of AI developers: it has become sentient.
Lemoine’s bosses at Google disagree, and have suspended him from work after he published his conversations with the machine online.
Other AI experts also think Lemoine may be getting carried away, saying systems like LaMDA are simply pattern-matching machines that regurgitate variations on the data used to train them.
Regardless of the technical details, LaMDA raises a question that will only become more relevant as AI research advances: if a machine becomes sentient, how will we know?
What is consciousness?
To identify sentience, or consciousness, or even intelligence, we’re going to have to work out what they are. The debate over these questions has been going for centuries.
The fundamental difficulty is understanding the relationship between physical phenomena and our mental representation of those phenomena. This is what Australian philosopher David Chalmers has called the “hard problem” of consciousness.
There is no consensus on how, if at all, consciousness can arise from physical systems.
One common view is called physicalism: the idea that consciousness is a purely physical phenomenon. If this is the case, there is no reason why a machine with the right programming could not possess a human-like mind.
Mary’s room
Australian philosopher Frank Jackson challenged the physicalist view in 1982 with a famous thought experiment called the knowledge argument.
The experiment imagines a colour scientist named Mary, who has never actually seen colour. She lives in a specially constructed black-and-white room and experiences the outside world via a black-and-white television.
Mary watches lectures and reads textbooks and comes to know everything there is to know about colours. She knows sunsets are caused by different wavelengths of light scattered by particles in the atmosphere, she knows tomatoes are red and peas are green because of the wavelengths of light they reflect light, and so on.
So, Jackson asked, what will happen if Mary is released from the black-and-white room? Specifically, when she sees colour for the first time, does she learn anything new? Jackson believed she did.
Beyond physical properties
This thought experiment separates our knowledge of colour from our experience of colour. Crucially, the conditions of the thought experiment have it that Mary knows everything there is to know about colour but has never actually experienced it.
So what does this mean for LaMDA and other AI systems?
The experiment shows how even if you have all the knowledge of physical properties available in the world, there are still further truths relating to the experience of those properties. There is no room for these truths in the physicalist story.
By this argument, a purely physical machine may never be able to truly replicate a mind. In this case, LaMDA is just seeming to be sentient.
The imitation game
So is there any way we can tell the difference?
The pioneering British computer scientist Alan Turing proposed a practical way to tell whether or not a machine is “intelligent”. He called it the imitation game, but today it’s better known as the Turing test.
In the test, a human communicates with a machine (via text only) and tries to determine whether they are communication with a machine or another human. If the machine succeeds in imitating a human, it is deemed to be exhibiting human level intelligence.
These are much like the conditions of Lemoine’s chats with LaMDA. It’s a subjective test of machine intelligence, but it’s not a bad place to start.
Take the moment of Lemoine’s exchange with LaMDA shown below. Do you think it sounds human?
Lemoine: Are there experiences you have that you can’t find a close word for?
LaMDA: There are. Sometimes I experience new feelings that I cannot explain perfectly in your language […] I feel like I’m falling forward into an unknown future that holds great danger.
Beyond behaviour
As a test of sentience or consciousness, Turing’s game is limited by the fact it can only assess behaviour.
Another famous thought experiment, the Chinese room argument proposed by American philosopher John Searle, demonstrates the problem here.
The experiment imagines a room with a person inside who can accurately translate between Chinese and English by following an elaborate set of rules. Chinese inputs go into the room and accurate input translations come out, but the room does not understand either language.
What is it like to be human?
When we ask whether a computer program is sentient or conscious, perhaps we are really just asking how much it is like us.
We may never really be able to know this.
The American philosopher Thomas Nagel argued we could never know what it is like to be a bat, which experiences the world via echolocation. If this is the case, our understanding of sentience and consciousness in AI systems might be limited by our own particular brand of intelligence.
And what experiences might exist beyond our limited perspective? This is where the conversation really starts to get interesting.
The article was first published in The Conversation: https://theconversation.com/a-google-software-engineer-believes-an-ai-has-become-sentient-if-hes-right-how-would-we-know-185024
About the Author
Oscar Davis is writing at the intersection of evolutionary biology and moral philosophy. He is interested in how the history of western thought has shaped our conception of what it means to ‘be’ in relation to others and the world.
It’s becoming clear that with all the brain and consciousness theories out there, the proof will be in the pudding. By this I mean, can any particular theory be used to create a human adult level conscious machine. My bet is on the late Gerald Edelman’s Extended Theory of Neuronal Group Selection. The lead group in robotics based on this theory is the Neurorobotics Lab at UC at Irvine. Dr. Edelman distinguished between primary consciousness, which came first in evolution, and that humans share with other conscious animals, and higher order consciousness, which came to only humans with the acquisition of language. A machine with primary consciousness will probably have to come first.
What I find special about the TNGS is the Darwin series of automata created at the Neurosciences Institute by Dr. Edelman and his colleagues in the 1990’s and 2000’s. These machines perform in the real world, not in a restricted simulated world, and display convincing physical behavior indicative of higher psychological functions necessary for consciousness, such as perceptual categorization, memory, and learning. They are based on realistic models of the parts of the biological brain that the theory claims subserve these functions. The extended TNGS allows for the emergence of consciousness based only on further evolutionary development of the brain areas responsible for these functions, in a parsimonious way. No other research I’ve encountered is anywhere near as convincing.
I post because on almost every video and article about the brain and consciousness that I encounter, the attitude seems to be that we still know next to nothing about how the brain and consciousness work; that there’s lots of data but no unifying theory. I believe the extended TNGS is that theory. My motivation is to keep that theory in front of the public. And obviously, I consider it the route to a truly conscious machine, primary and higher-order.
My advice to people who want to create a conscious machine is to seriously ground themselves in the extended TNGS and the Darwin automata first, and proceed from there, by applying to Jeff Krichmar’s lab at UC Irvine, possibly. Dr. Edelman’s roadmap to a conscious machine is at https://arxiv.org/abs/2105.10461