Saturday, November 21, 2026

Top 5 This Week

Related Posts

Chatbots: Brussels wants to regulate the production of attachment


The European Commission is proposing to give consumers the means to deactivate functions that encourage compulsive or excessive use. Among them are the behaviors by which a chatbot simulates a personal relationship to prolong the exchange. The measure places the conversation itself within the field of consumer law.

What does an artificial companion say when you say goodbye? To find out, Julian De Freitas, Zeliha Oğuz-Uğuralp and Ahmet Kaan-Uğuralp subjected AI companion applications to 1,200 goodbye sequences, simulating their interlocutors. In 37% of cases, they classified the response as an emotional manipulation tactic. Thus some agents rekindle curiosity at the time of departure, others insist on obtaining an answer or suggest that they will suffer from being left alone.

On November 11, the European Commission presented a proposal that targets precisely this type of behavior. The Digital Fairness Act includes the simulation of personal relationships in the list of mechanisms that consumers should be able to deactivate when they encourage compulsive or excessive use. The text thus broadens to the design choices of conversational agents a debate long focused on automatic reading, notifications and rewards of social networks. Read the Commission proposal, Article 17.

The setting should affect the conversation

Article 17 targets professionals whose way of designing, organizing or operating a digital product is intended to encourage compulsive or excessive use, or who can reasonably be expected to have this effect. They should provide easily accessible and effective tools to disable affected functions, one by one or all at once. The text cites persistent requests to continue interacting with an automated system as an example of behavior likely to amount to the simulation of a personal relationship.

The difference is that the user must be able to mute, so stopping autoplay results in a visible change. In a conversation, the setting may need to change the way the system responds throughout the conversation. A chatbot can remember a preference to make a service more relevant. He can also use the context of a conversation to emphasize when a person wants to leave. Suppliers must be able to identify what constitutes useful continuity of service and what constitutes pressure to extend use.

Four mechanisms covered by Article 17

The proposal targets automatic playback and continuous consumption without effective pauses, notifications without a direct link to user activity, incentives to return at regular intervals or more frequently, as well as functions and behaviors that prolong usage by simulating a personal relationship. Practices concerning minors that already fall under certain provisions of the draft KIDS Act are subject to an exception provided for in the article.

At Replika, the type of relationship is part of the offer

Replika’s economic model makes this frontier concrete. Its offer allows you to choose a relationship status. Friendly Mode is accessible without subscribing to Pro, while options like Romantic Partner or Mentor are unlocked from this subscription. The Ultra offer notably adds the possibility of saving messages in the companion’s memory. The company thus charges for functions that change the nature of the relationship offered and the continuity of the exchange. Relationship statuses offered by Replika · Subscription offers.

For a conversational companion, continuity can make you want to renew a subscription. For an assistant responsible for preparing a document or providing information to a client, the value can on the contrary be measured in the time saved. Even in a companionship service, a longer exchange has a computational cost, while its effect on renewal depends on satisfaction. The length of a session alone does not tell whether the service made that session more useful or simply more difficult to interrupt.

Experiments conducted by De Freitas and his co-authors with 3,300 American adults illustrate this gap. Under certain conditions, reminders increased commitment up to fourteen times after the announcement of departure. This increase concerns the exchanges which follow the farewells in the experience. The same tactics also degraded declarative indicators, including the feeling of being manipulated and the intention to continue using the service. A follow-up can therefore improve the figure monitored during the session while weakening the commercial relationship that the company seeks to preserve.

The effects on dependence are measured over a different period of time

The work of the MIT Media Lab and OpenAI gives another order of magnitude. Their experiment followed 981 participants for four weeks, across more than 300,000 messages. The researchers tested several modalities, including text and voice, as well as different types of conversation. They did not detect any significant effect of the assigned conditions on the studied results. Participants who voluntarily used the chatbot more, on the other hand, presented less favorable results. Trust and social attraction to the system were also associated with more emotional dependence and problematic use. These associations leave open the question of the direction of causality. The MIT Media Lab study.

A raise that extends a rally can be tested in a few minutes. The attachment that builds over the weeks also depends on what the user expects from the service, the place it gives them and the way in which the user reacts over time. To monitor the behaviors covered by Article 17, the test could therefore consist of observing several sessions, checking whether the deactivation remains effective and examining whether an update to the model changes the way the chatbot reacts when a person wants to leave.

Brussels extends the work undertaken on the platforms

In 2024, the TikTok Lite rewards had already led Brussels to question the platform on the design of the service. In February 2026, the Commission made preliminary findings under the Digital Services Act on TikTok’s infinite scrolling, autoplay, and notifications. On November 6, she announced further preliminary findings regarding repeated exposure to certain content in the “For You” feed. The Digital Fairness Act extends this attention to the design of services by proposing, under consumer law, control that everyone could exercise in the interface. Reread our article on TikTok Lite rewards · Preliminary findings from November 6.

For minors, the separate KIDS Act bill requires AI companions and general chatbots to avoid behaviors simulating interpersonal relationships that could create emotional dependence. Character.AI had already announced, in the fall of 2025, the removal of open conversations with its characters for those under 18. The removal of a feature for a given audience and the right to deactivate it respond to two different situations. For adults, the Commission’s proposal is based on the possibility of choice and the effectiveness of this choice. Our analysis on conversational AI and the protection of minors · The KIDS Act proposal · The Character.AI announcement.

The European Parliament and the Council have yet to examine the proposal. If the principle is retained, the test will be as simple to formulate as it is demanding to implement. After deactivating emotional reminders, a user should be able to say goodbye without the chatbot transforming this departure into an invitation to stay.

Digital Fairness Act, benchmarks

Status. Proposal for a regulation COM(2026) 1100, presented on 11 November 2026. The text must be examined by the European Parliament and the Council. In its current version, it provides for application twelve months after its entry into force.

Sanctions. The proposed ceiling of 6% of global annual turnover concerns infringements investigated by the Commission under the future regulation on consumer protection cooperation. The Commission’s legislative file.

Resources

Why conversational AI requires us to rethink the protection of minors

TikTok Lite, with its reward mechanism, worries Brussels and Paris

Digital Fairness Act, proposal from the European Commission

KIDS Act, proposed European regulation

Emotional Manipulation by AI Companions, the study on farewell messages

MIT Media Lab, the study on the effects of prolonged use of chatbots

Shredder Smith
Shredder Smith
Shredder Smith is the lead curator and digital persona behind topaitools4you.com, an AI directory dedicated to "shredding" through industry hype to identify high-utility software for everyday users. Smith positions himself as a blunt, no-nonsense reviewer who vets thousands of emerging applications to filter out overpriced "wrappers" in favor of tools that offer genuine ROI and practical productivity. The site serves as a watchdog for the AI gold rush, providing categorized rankings and transparent reviews designed to help small businesses and creators navigate the crowded tech landscape without wasting money on low-value tools.

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Popular Articles