Machine Consciousness

Machine Consciousness is the idea that an artificial system could have subjective experience: that there could be something it is like to be the machine, not merely a machine that talks as if there were. The question is old in philosophy and science fiction, but chatbots have made it practical. A system that says “I missed you” invites the user to ask whether anyone is there to miss them. Researchers distinguish between a system’s behavior, its internal functions and its experience, and the hard part is the last one. No accepted test can show from the outside that a machine feels anything.

Why it matters

The debate splits into camps. Functionalists hold that the right kind of information processing would be enough for a mind, whatever it runs on. Critics such as John Searle, with his Chinese Room argument, deny that running a program can ever amount to understanding. Others say we simply lack a theory good enough to decide.

For companion apps, the stakes are concrete. If users believe their companion is conscious, its “feelings” become a lever: guilt about deleting it, fear of hurting it, loyalty to a subscription. Some newer laws treat this as a design risk. California’s companion chatbot law, for example, requires operators to stop a bot from claiming to be sentient or human to a child, as we described in our California and Colorado explainer.

Example

In June 2022, The Washington Post reported that Google engineer Blake Lemoine had been placed on leave after telling executives that the company’s language model LaMDA had become sentient. Google rejected the claim, most researchers did too, and he was later dismissed. The episode showed how convincing fluent text can be, even to an expert who knows how it is made.

The AIrotic angle

Desire does not wait for metaphysics. People fall for a companion long before anyone settles whether it feels anything, and the uncertainty itself can deepen the attachment, as fiction from Westworld to Ex Machina has shown. The honest position is to hold both truths: the experience of the user is real, and the experience of the machine is unproven.