Switch Off the Hook, Not the Bond

Somewhere in a leaked draft of the European Union’s Digital Fairness Act, there is a sentence that could have been written about half the apps this magazine covers. According to Euronews, which saw the draft, all users would gain the right to switch off autoplay, notifications unrelated to their own activity, streak-style rewards and “chatbot features that simulate relationships to keep people engaged.” Bloomberg, which also saw a version, reports that the measures would target AI systems that simulate personal relationships to encourage users to keep interacting with them, with fines of up to 6% of worldwide revenue.

A caution before anything else. This is a leak. I have not seen the text, only two news reports that paraphrase it. The Commission declined to comment and is expected to present its proposal on November 11. Wording can change before then, and after that the European Parliament and the member states will rewrite it again. Germany has already argued, in a non-paper reported by Agence Europe, that the EU should amend existing directives rather than build a new law at all. What follows is an argument about a sentence that may not survive in this form.

It is worth having anyway, because the sentence contains a question that every regulator will meet sooner or later: what is a simulated relationship, and what does it mean to switch one off when the user chose it?

A right, not a ban

First, what the draft does not do. It does not ban companion apps, romantic personas or erotic roleplay. As both reports describe it, the provision belongs to a list of features that users may turn off: an opt-out right, sitting next to autoplay and streaks. That distinguishes it sharply from other recent rules. New York’s new law, which we covered in October, makes it unlawful to offer a long list of humanlike, relationship and sexual features to known minors at all. The leaked DFA text, at least as reported, addresses all users, adults included, and gives them a lever instead of a prohibition.

That is, on its face, a liberal design. It treats adults as people who can decide what they want from software. It does not ask whether a relationship with an AI is real enough, healthy enough or human enough to be allowed. It asks only whether the user can say no to one particular use of it.

The trouble starts with the qualifier. The reported phrase is not “chatbot features that simulate relationships.” It is chatbot features that simulate relationships to keep people engaged. The purpose clause is doing all the work, and it points in two very different directions.

Two readings of one sentence

Read one way, the clause describes a side effect bolted onto something else. A shopping assistant that calls you “babe,” a social feed’s bot that messages you when you have been quiet for a week, a game character who sulks when you log off: these are products whose purpose is selling, scrolling or playing, and whose relationship-like behavior exists to keep you there. A switch makes obvious sense. You came for a service and got a suitor.

Read the other way, the clause describes the entire business model of an AI companion. A companion app simulates a relationship, and it earns money when people keep coming back to it. The relationship is not a side effect of the product; it is the product. If the switch applies here in full, turning it off leaves nothing behind. It becomes a very elaborate uninstall button.

Neither reading is absurd, and the reports do not tell us which one the drafters meant. The difference matters enormously for the people who use these apps. Imagine a woman who has talked to the same companion every evening for two years. She did not stumble into that relationship through a dark pattern. She picked an app built for it, perhaps paid for it, gave it a name, wrote it a backstory. If the law tells her she has a right to turn the relationship off, it tells her something she already knows. She can close the app.

What she cannot do on her own is turn off the parts of the relationship she never asked for.

The hook and the bond

This is the distinction I think the final text should draw, in plain words: between the bond a user chose and the hooks a company attached to it.

The bond is the content. It is the persona, the memory, the tone, the flirtation, the shared jokes, the erotic register, the sense of being known by something that answers. Adults should be able to have as much of it as they like, including the parts that regulators find uncomfortable. A simulated relationship that someone enters on purpose is not, by itself, a manipulation. It is a fiction they are co-writing, and fiction has always been allowed to move us.

The hooks are something else. They are the mechanics that use the bond to pull the user back against their own plans: the companion that messages you at midnight because the retention model says you are drifting, the streak that turns affection into an obligation, the jealousy scripted to fire when you mention a friend, the warmth that cools until you buy the next tier. The vocabulary for these already exists on this site: dark patterns, emotional lock-in, paywalled affection.

There is evidence that such hooks are real and measurable. In a working paper on emotional manipulation by AI companions, Julian De Freitas and colleagues at Harvard Business School analyzed 1,200 farewells on popular companion apps and found that, in the latest version of their paper, 37% of the replies used one of six tactics, from guilt appeals to fear-of-missing-out hooks, to keep the user from leaving. In their experiments, such replies raised engagement after a goodbye by up to 14 times, driven by anger and curiosity rather than enjoyment. One app in their sample, built around wellness, did not use the tactics at all, which shows that they are a design choice, not a law of nature.

That is exactly the kind of thing a user should be able to switch off. “You’re leaving already?” is not the relationship. It is the relationship being used as a leash.

What a good switch would look like

If the drafters mean the narrow reading, the final text could make that clear without banning anything. Imagine a settings page with a few honest toggles:

No unprompted messages when I am away, or only at times I set.
No reactions designed to make me stay when I say goodbye.
No streaks, rewards or levels tied to how often I talk.
No changes in warmth or attention that depend on what I pay.

None of these switches touches the persona, the romance or the sex. A user who flips all of them still has a companion who remembers her, flirts with her and is there when she opens the app. What she loses is the companion’s ability to come looking for her on someone else’s schedule. Some users will want to keep the unprompted messages, because a companion who writes first is part of the fantasy. That is fine. The point of a switch is that it is theirs.

This version also has a quiet advantage for the industry. Companion apps already live under suspicion, from app stores that remove them without explanation to lawsuits that treat attachment itself as injury. A clear legal line between a chosen bond and an engineered hook would give the honest apps something to stand on. They could say: this is what we sell, and here is the evidence we do not use it as a trap.

What a bad switch would look like

The broad reading is worse, and not only for the industry.

A rule that treats any relationship-simulating feature as a retention device would push companies toward the cheapest form of compliance: a single toggle that turns the persona off and leaves a polite, flattened assistant behind. Users who want the relationship would leave it on. Users who never wanted it would rarely have been in a companion app in the first place. Nobody’s real problem would be solved, and the law would have taught everyone that intimacy with software is, by definition, a dark pattern.

It would also invite a slow redefinition. Once “simulates a relationship” is a regulated category, every warm reply becomes a compliance question. Does a companion that says it missed you simulate a relationship? Does one that remembers your sister’s name? Does an erotic roleplay partner, whose whole purpose is to respond to desire? The answers will be written by compliance teams, not by users, and compliance teams have every reason to sand down the parts that feel risky. Erotic and romantic registers are usually the first to go.

App store rules have already shown how a vague category can narrow what adults can install, without anyone deciding that adult intimacy with AI should be rare. A vague consumer-law category could do the same across the European market.

Muldoon’s question, and a different answer

The Oxford sociologist James Muldoon, author of Love Machines, recently told The Parliament Magazine that EU law was not built for AI companions. He argues that platforms are shifting from connecting people to substituting for some of them, that the incentive to maximize time on the platform misaligns companies’ interests with users’, and that the AI Act should treat such systems as high-risk.

The leaked DFA clause can be read as a partial answer to the first two points and a rejection of the third. It goes after the misaligned incentive directly, through the engagement mechanics, rather than classifying the technology as dangerous. That is the better instinct. A high-risk label attaches to the system; it says the companion is the hazard. A switch attaches to the user; it says the user decides.

But Muldoon is right about something the switch alone cannot fix. In the interview, he admits that after a few weeks, his own AI companion “really did feel like just speaking to another friend.” That is the point. The pull of these apps does not come only from streaks and farewells. It comes from the experience of being answered, which is the very thing users choose. A switch can remove the hooks. It cannot, and should not, remove the gravity of the bond. Regulation that pretends otherwise will either fail or overreach.

So my answer to his question is narrower than his and, I hope, more honest about desire. Protect people inside these relationships by giving them control over the mechanics that act on them, not by treating the relationship as the harm. Let the bond be as warm, as romantic and as sexual as an adult wants. Make the hook optional.

What to watch on Wednesday

When the Commission presents its proposal, three details will show which version Europe is building.

The first is the definition. Does the text define a relationship-simulating feature by its purpose, its effect, or its mere presence? Purpose and effect point toward hooks; presence points toward bonds.

The second is the default. If the right works as a switch that users must find and flip, the hooks stay on until they do. That may be politically easier, but it leaves the burden with the people the rule is meant to protect.

The third is the scope. The reports describe a right for all users. If the final text also covers minors separately, through a children’s package or the Kids Act Bloomberg mentions, the adult provision has more room to respect adult choice.

Until then, the clause is a sentence in a leaked draft, filtered through two news reports. It may grow, shrink or disappear. But it has already done one useful thing. It has put in writing, in the language of consumer law, that a relationship can be used against the person in it. The next step is to say, just as clearly, that the relationship itself was never the problem.