As the end-of-year holidays approach, Emmanuel Macron wants to ban certain uses of AI when they pose a risk to children. This particularly concerns chatbots, but also connected toys. In an interview with Les Échos, he discusses these dangers and specifies that “it is not because it is AI that it must be the Wild West”.
A current system that is not sustainable
This declaration comes on the eve of the Paris Peace Forum, when a charter must be adopted, notably by OpenAI and Anthropic. The head of state believes that the current system is not relevant. It is imperative to change it, or even prohibit certain uses if they present risks. Concerning protection against AI, Emmanuel Macron received this Tuesday the recommendations of a committee of experts.
As for the charter, it must bring together AI companies like OpenAI and Anthropic to strengthen the security of their systems. The latter must in particular accept the “principle of a pre-audit and pre-certification with a prior study of the models before putting them in the hands of children”.
Toys designed with AI are worrying
The Christmas holidays are approaching, so AI toys will undoubtedly end up under the tree, but they worry the head of state. The latter had notably asked Raphaël Gaillard and Amine Benyamina, two psychiatrists, to study the risks linked to these technologies. The main objective was to know if conversational agents could promote unrestaddiction or even damage to the mental health of children.
While video games are often singled out for violence, it is now AI toys that find themselves at the center of the debate. These two psychiatrists are also part of the committee responsible for making recommendations on these AI systems. Its report, submitted this Tuesday, recommends in particular better regulating the mechanisms likely to exploit the vulnerabilities of the youngest. We will therefore have to wait to find out if certain uses will be prohibited.
Toys already withdrawn from the market
Last year, Kumma had been withdrawn from sale by its manufacturer. Powered by GPT-4o, the plush had undergone some tests. These were not conclusive since she notably suggested sexual role-playing and indicated where to find knives in the house. The Head of State therefore does not want potentially dangerous AI systems to fall into the hands of children without having been first evaluated.
In European law, such a ban already exists for certain practices. Indeed, the AI Act prohibits AI systems that exploit a person’s vulnerability due to their age, when they materially alter their behavior and cause, or are likely to cause, significant harm. To go further, Emmanuel Macron wants to bring regulation to the European level.

