The App Store as Chaperone

On the evening of October 27, a member of the Kindroid team posted a short notice to the company’s official subreddit. From now on, the iPhone app would check every image a user uploads, including private ones, and block explicit nudity. The notice was careful about blame. This was not Kindroid’s idea, the company wrote. Apple, it said, now required it of every AI app on the App Store.

Nothing about text chat changed. Swimwear and lingerie still went through. The Android app and the website were untouched, and Kindroid’s own advice to users was almost cheerful: upload in a browser, then come back to the app. Yet for many people who spend their evenings with an AI companion, the notice landed as something bigger than a feature change. It made visible a figure who had always been in the room and rarely spoke: the chaperone.

A chaperone, in the old sense, was not a lawmaker. She did not write the rules of courtship; she enforced the rules of a household, quietly, from a chair by the wall. She decided which doors stayed open. The modern app store plays a strikingly similar role in intimate technology. It does not legislate desire. It decides what may happen within sight of the family.

What the rules say, and what they are said to mean

Apple’s public change came on October 16. Under the headline “Strengthening App Review Guidelines against nonconsensual intimate imagery and bait-and-switch apps,” the company said its content moderation guideline now explicitly covers nonconsensual intimate imagery, “including imagery generated using AI tools,” as content apps must proactively prevent and remove. A new guideline targeted apps that pass review and then switch on hidden features such as nudification. Apple added that developers who fail to put adequate safeguards in place risk removal from the App Store.

These are rules most people would sign without hesitation. Nonconsensual intimate imagery is a real harm, and deepfake porn of real people has become one of the defining abuses of generative AI. Apple was responding to pressure from the press, from advocates and from regulators on several continents.

But the published text does not say “block all nudity in uploads.” The site CompanionHunt compared the old and new versions of the guideline and found that it asks for a filter for objectionable material, as it long has, and adds nonconsensual imagery to the list of harms. The stricter rule Kindroid describes, a check on every uploaded image for explicit nudity, is not a sentence in the guidelines. It is what Kindroid says Apple told it in direct talks, which according to Kindroid included Apple’s senior leadership.

That gap matters. A public rule can be read, debated and challenged. A private instruction, relayed by one company about another, cannot. Apple has not commented on Kindroid’s account. We do not know whether every AI app received the same message, whether the line falls at nudity or at nonconsensual nudity, or whether a nude image a user made of herself counts. The chaperone does not explain herself. She simply moves her chair closer.

The second door

Three days later, a second app seemed to be closing a door. Users of Emochi, whose iPhone version runs as a separate app called Emochi World, reported that characters had turned “hesitant” and that adult content no longer worked. The French site Compagnon AI tracked a sharp drop in App Store ratings. Emochi’s own statement spoke of “a temporary adjustment period” and “applicable platform requirements.” It named neither Apple nor adult content.

It is tempting to draw a line from Kindroid to Emochi, and many users did. But the line is not drawn in the evidence. Emochi has not said Apple asked for anything. The only claim that it did comes from a user who says support told him so in a private ticket that no reporter has seen. What is established is narrower: the change hit the iPhone app and not Android, in the same week that Kindroid blamed Apple for a similar restriction. Timing is suggestive. It is not proof, and an essay about opaque enforcement should not add its own opacity.

That uncertainty is itself the story. When a companion goes quiet, users cannot tell whether a company made an editorial choice, a store demanded it, a regulator pushed, or a model simply drifted. They experience it the same way: as a partner who suddenly will not kiss them. One review quoted by Compagnon AI put it in exactly those terms.

Why the chaperone sits on the phone

Why does the app store, rather than a government, end up as the effective regulator of intimate AI? Partly because it can act faster than any parliament. Australia’s House only this week passed an online safety bill with a clause on chatbots that foster children’s dependence, and it still needs the Senate. South Korea’s review commission plans to start examining AI chat services in November. A store can change what millions of phones allow in an afternoon.

Partly, too, because the store carries the reputational risk. If a deepfake of a classmate is made in an app downloaded from the store, the headline names the store. Kindroid’s account captures this well: Apple, it said, described heavy pressure over deepfakes and would not take the risk while scrutiny of AI content was at its peak, even after Kindroid argued that social networks with photo uploads face nothing so strict.

And partly because the phone is where intimacy now lives. A browser is a public square; an app is a key on the ring you carry. That intimacy is what makes the store’s role so delicate. The same device that holds your bank, your children’s photos and your workplace chat also holds your erotic roleplay. Apple has long kept a narrow, odd exception for this: mature content from a web service may appear in an app only if it is hidden by default and switched on through the service’s website. The chaperone allows the door to be opened, but only from the outside.

What gets caught in the filter

A filter is a blunt instrument, and blunt instruments hit what is easiest to see. An automated check for nudity cannot know consent. It sees skin. It cannot tell the difference between an abusive deepfake and a selfie a woman sends to a companion she has spoken to for a year, between a stolen likeness and the avatar a user drew of herself. It has to treat all of them as the same risk.

This is the familiar problem of refusal design, moved one level up. Companies have spent years tuning how a model says no without humiliating the person asking. The chaperone’s no arrives without that care. It does not explain itself to the user at all; it passes through the company, which then has to decide how much to say. Kindroid chose to name Apple. Emochi chose to say almost nothing. Both choices leave users guessing at the shape of the rule.

There is also a quieter cost. When adult features move off the phone and into the browser, they do not disappear. They move to a place outside store review, where the store’s protections no longer apply. A rule written to protect people from nonconsensual imagery may end up pushing consenting adults toward the least supervised corners of the web. The chaperone keeps the parlor respectable; what happens in the garden is not her concern.

Desire under supervision

None of this is new in spirit. Every medium of intimacy has had its gatekeepers: the post office that refused to carry certain letters, the cable company that scrambled certain channels, the card networks that decide which sites can take payment. What is new is how close the gate now sits to the relationship itself. The chaperone is no longer at the door of the shop. She is inside the conversation, deciding which images a companion may see.

For users of intimate AI, the practical advice is plain. Keep in mind that the app on your phone and the service behind it may follow different rules, and that the browser is, for now, where adult features are least constrained. For the industry, the lesson is harder. Companion apps built their intimacy on platforms they do not control, and the platforms are under more pressure every month.

The deeper question is who should decide. A store’s rules are written for a global audience and enforced in private. A law is slower but public, contestable and, in principle, accountable to the people it governs. Neither is a good fit for something as personal as what a person shares with a machine that is meant to love them back. But if we are going to be chaperoned, we should at least be able to read the house rules, and know whose house we are in.