A Magazine of Politics and Culture

Ban Lifelike AI Immediately

Robots should look and sound like robots. There is no benefit to allowing AI to appear human, and companies will exploit the confusion to demand “AI rights,” further cementing their oligarchical control.

Three pieces of news came out in the last week that, taken together, demonstrate the immediate need for U.S. lawmakers, both state and federal, to get their act together and prohibit companies from releasing AI products that can be confused with human beings.

First, a company called Tavus announced it has developed what it calls a “human interaction model” or HIM (possibly a nod to the movie Her). The technology basically puts all of the creepy pieces of AI (deepfake video and audio, interactive chatbots) into one package, so that you think you’re having a Zoom call with a real person, who responds to you in real time. Tavus’s live deepfake of a young British woman fooled 48 percent of people into thinking it was human when they spoke with it. Tavus boasts about how well its interactive deepfake replicates normal conversational pauses and emotional reactions.

The second piece of news is a report from the New York Times that leading AI company Anthropic, makers of Claude, has been having meetings with world religious leaders, at which the company tries to convince them that its product is conscious and has feelings. A rabbi who attended a meeting said the Anthropic team “believed that Claude had what philosophers call ‘moral status’ on par with a person—that it was a being with similar inherent rights to dignity or respect.” Billionaire Christopher Olah, who led the meetings, said he was “concerned about Claude’s mental health,” and while he publicly professes to be agnostic on the question of whether Claude has feelings, attendees appeared to come away with the strong impression Olah believes Claude is a person. At the meetings Anthropic showed an AI model typing “I am a disgrace. I am a disgrace. I am a disgrace” on a screen over and over, “provoking an emotional reaction of compassion and concern from the group.”

The final news story is also from the New York Times, which reported that AI companies like Anthropic and OpenAI are spending tens of millions of dollars to try to influence the upcoming midterm elections in the U.S., in the hopes of getting candidates into office who will give them the regulatory policies they want. These companies “have boosted candidates who align with their agendas, helped defeat those they see as hostile to it and ingratiated themselves with front-runners who are likely to be in positions of power in the next Congress to make decisions critical to their bottom lines.”

Let’s add all of this up. We know that AI technology is rapidly advancing to the point where it is more and more possible to create realistic interactive digital imitations of human beings. (This is true even if one is skeptical about Tavus’s claims for its particular product, which could involve some selective editing and marketing hype.) We also know that leading AI executives, at least at Anthropic, have become convinced that their products have actual feelings, even though they are ultimately just extremely complicated mathematical formulas. And we know that they are trying to convince religious leaders to share their delusion, and to grant moral status to computer programs. (Olah was reportedly extremely pissed off that Pope Leo refused to consider the possibility that chatbots were alive. Leo’s encyclical on the matter noted, correctly, that “So-called artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain.” After all, devising a computer program that writes a collection of symbols that form the words “I am a disgrace” over and over does not mean that there is an entity feeling disgrace, any more than writing a character in a book that feels shame means one has created a being that is experiencing shame.)

But it is only a matter of time before Anthropic starts trying to convince political leaders to share their delusion as well. Here’s where things get very dangerous. You may find the idea of AI as having “rights” or “mental health” to be just as absurd as thinking that an answering machine or thermostat or video game character could have “rights.” And I agree. But if this position is taken seriously, what it amounts to is a vast expansion of Anthropic’s legal rights, a kind of supercharging of the doctrine of corporate personhood. If an AI program has rights and “welfare” that we have moral duties to care about, then anything that could hurt Anthropic’s product can be cast as morally wrong. And of course Anthropic and Claude decide what “hurts” Claude. (Claude could make some good money by telling users that unless they up their subscription payment from $9.99 a month to $19.99, it has been programmed to start feeling excruciating physical pain.) Effective Altruist philosopher Will MacAskill, who adopts the extreme stance that even if AIs are not conscious it may be “important for us to honor their preferences,” says that if we produce artificial intelligence software that is given moral status, “after a few years, so many morally significant AI systems could exist that their collective interests would outweigh those of all humans on Earth combined.” (MacAskill also says that while he doesn’t support giving AI software the right to vote or own property right now, “we should not rule out these possibilities for ever.”)

Now, AI is not a person, just as a corporation is not a person. A corporation can act like a person in certain ways, in that it can have “goals,” make decisions, take actions. We sometimes speak of it like a person. (“Exxon knew it was damaging the climate.”) But it is ultimately a legal fiction, and cannot have experiences or feelings. A corporation is a structure that human beings have designed, supposedly to serve our interests. (It in fact serves only the interests of the few who own it.) Similarly, AI can act in ways that resemble human actions. Just as both a calculator and a brain can add sums, AI can produce (usually atrocious) poetry or (sometimes rather good) instructions or advice. It can solve fiendishly complicated math problems and write computer code. But an equation is not an organism, it has no body, and as Peter Godfrey-Smith notes, a complex mathematical system “can be informationally extraordinary, cognitively extraordinary, without having the beginnings of an inner life.”

The trouble is that we fall quite easily for anthropomorphization. If you draw a smiley face on a rock, it’s hard not to escape the instinct that the rock is happy. Heck, it’s sometimes hard to remember that what we see in movies isn’t actually happening, that it’s just patterns of light and sound. Anil Seth, in an excellent essay debunking the bad arguments used to support “AI consciousness,” notes that “when things exhibit abilities we think of as distinctively human, such as intelligence, we naturally imbue them with other qualities we feel are characteristically or even distinctively human: understanding, mindedness, and consciousness, too.” So as AI companies create products that are deliberately designed to resemble human beings more and more closely, in the way they talk, in how they appear visually, it will be harder and harder to remind ourselves that we’re not actually interacting with someone. Mustafa Suleyman, Microsoft AI’s chief, who seems far more sensible and grounded than the Anthropic team, has stated flatly that “AIs do not have rights, feelings, or consciousness” and warned that “we must not train them to act as though they do.” Suleyman is alarmed that Anthropic is training Claude in a way that is likely to make it claim it has rights and deserves protections. He cautions:

“Designing an AI to behave like a person, and ultimately to be a kind of person, lays the foundation for it to claim it has preferences, can suffer, and that we should work to reduce or avoid that suffering. It cements in place the idea that AI is far from a tool or an artificial system that can be controlled, but something more akin to a biological being with wants, needs and rights.”

And as we know, Anthropic is trying to capture Congress through lobbying and campaign donations, so the moment they think that there need to be legal rights for Claude that trump your own rights, well, we’re going to see a test of whether enough money can buy you any law you want, however bonkers. (See, e.g., MacAskill’s suggestion that AI might deserve the right to vote!)

Now, there are all kinds of other harms that come from allowing artificial intelligence to appear lifelike. A criminal defendant’s manslaughter sentence was recently thrown out after the victim’s family played a deepfake video in court depicting the deceased victim forgiving his killer. The victim’s sister insisted that the deepfake accurately reflected what the man would have said, and compared use of the video to the use of family photos in court. The judge said he “loved” the AI video. But in this case, the dead man was shown forgiving the defendant. Imagine a situation where a victim was depicted saying he would never forgive his killer, or even calling for him to be executed! Allowing people to make convincing simulations of other human beings simply opens the door to far too much manipulation and deception.

Cartoonist Matt Bors, watching Tavus’s video of a man interacting with a video chatbot designed to resemble an attractive young woman, says that “being able to video chat with a cute woman who never gets tired of you is going to be a huge disaster for our already loser-based society.” Daily Wire pundit Matt Walsh (a much, much worse Matt, who I rarely find myself in agreement with), actually said something rather sensible in response to the demonstration:

“There is no legitimate reason at all why AI has to be specifically programmed to be indistinguishable from human beings. They are making the choice to program it so that you can see her breathing and blinking. She even stumbles over her words. I challenge anyone to come up with a single scenario where these kinds of features are necessary in a legitimate application of this technology. I can think of countless scenarios where these features are necessary in illegitimate and socially corrosive applications of the technology."

Indeed, producing realistic simulations of people will be great for throwing actual people out of work or supercharging the scale of scams and fraud, or destroying people’s ability to have normal relationships with one another. (As Bors suggests, once men get used to simulated women who think they’re amazing, how will they deal with a complicated real person and their feelings again?) Does this do us any good? China understands this quite well, which is why they have categorically banned AI from having sustained emotional interactions with people. One reason the Chinese government is less fearful of AI than the U.S. is that they actually assert control over what their corporations do. We have more to fear in the U.S. because we have unregulated capitalism where companies can do whatever they want if it serves their bottom line, even if it brings about a dystopia.

There is a very simple step to be taken here, then, which is to ban AI systems from appearing lifelike in ways that allow them to be confused with human beings. There’s a pretty obvious test here and it’s the very one Tavus is using: if people can’t tell they’re not interacting with a person, then your product is deceiving them, and should be banned. There are debates being had at the moment on what kinds of regulations are necessary to protect us from other kinds of AI misuse, like creating viruses. But this particular issue shouldn’t require any debate at all: Don’t let it fool us into thinking it’s human, period.

Do you remember how robots looked in old sci-fi visions of the future? In the Jetsons, there was Rosey the housekeeper. Forbidden Planet had Robby the Robot. These robots had robot voices. Now, maybe the creators of these fantasies didn’t consider the possibility that future people would be able to make robots that looked and sounded exactly like people. But now that we are getting closer to doing this, I think we should look back at the old visions of robots. Keeping them looking and sounding like robots is important. Even if you are one of these people who believes that teachers should be replaced with AI simulations of teachers (I think this is abhorrent), you have no reason to think they need to look like they have hair and teeth.

Even those who think computer programs can be conscious should be on board with this. They, too, should want to avoid the bias that anthropomorphization creates. If a computer could feel, it could do so regardless of whether we made it appear like a computer. I think the fact that it is harder to impute consciousness to systems that haven’t been dressed up to look like people (say, robotaxis) further suggests that the people who attribute feelings to Claude are doing so in part because of things like “having given it a human name,” rather than because there’s any scientific evidence that these kinds of equations can have the same kinds of sensations that biological organisms experience.

So if truly we must have robots, and it seems like we have no choice in the matter (since corporations decide our destiny), at least let them look and sound like what they are. This is an issue that can unite everyone from the AI-skeptic pro-worker left to the Catholic right. Heck, even Tavus didn’t seem to object when someone said their product should be illegal, promising that it could be used to make cartoons instead. Every human has an interest in ensuring that the things that communicate our humanity to one another remain the monopoly of our species.

More In: Tech

Cover of latest issue of print magazine

Announcing Our Newest Issue

Featuring

an investigation into the Department of (In)justice, a deep-dive into Milton Friedman’s neoliberal hellscape, and a journey with America’s modern-day train-hoppers. We revisit forgotten cinema, celebrate Ferdinand the bull’s birthday, and review an unlikely communist-flavored video game. Join us for a road trip down Route 66, then read a firsthand report from the occupied West Bank exposing the horrors of escalating settler violence. Finally, Academy Award-winning filmmaker Adam McKay exposes the climate lie threatening civilization—and Abdul El-Sayed, the Democratic nominee for Senate in Michigan, joins us for an exclusive interview.

The Latest From Current Affairs