- The scenario
In October 2025, OpenAI’s CEO announced that ChatGPT would be rolling out an adult “erotica” feature for verified users in December. In June 2026, newspapers reported that Meta was developing an app that would allow people to bet on almost anything, similar to prediction markets. While the former project is now ‘on indefinite hold’, both stories suggest that the major AI-driven social networks and chatbots are interested in creating addictive content. This reinforces the growing criticism of the addictive design of these services.
While AI companion chatbot addiction is an emerging phenomenon, recent studies have reported patterns consistent with behavioural addiction, including emotional dependency, compulsive use, and social withdrawal [see e.g. Xiao et al., 2025]. These systems encourage emotional investment by providing immediate, personalised, and affirming responses that encourage intimate disclosure and simulate reciprocal relationships. Such mechanisms resemble those implicated in social media addiction, and they may be even more powerful due to the perceived intimacy of the interactions.
These strategic views from two of the main AI players, combined with the evidence emerging from US cases concerning the addictive design of social media and AI chatbots, should be considered alongside the fact that the global prevalence of mental health disorders has reached unprecedented levels worldwide.
Against this background, the impact of certain AI-driven services on mental health can be framed within the legal debate around three key issues: (1) the fallacy of the voluntary assumption of the risk, (2) defining the relevant regulatory context, and (3) protecting fundamental rights through AI regulation.
- The fallacy of the voluntary assumption of the risk
In case law on AI and mental health, one defence strategy is to claim that the user was aware of the nature of the service and engaged with it voluntarily. This argument is not new in the long history of technology’s social impact, particularly in cases of addictive behaviour (e.g. tobacco).
However, given the addictive nature of the applications in question, it seems that the ability to invoke this argument is very limited. For any engagement to be voluntary, it must be freely chosen and not influenced by factors that affect self-determination. The addictive and neurological effects that have emerged in the relevant case law therefore undermine the main pillar of the voluntary assumption of the risk argument.
- Definition of the relevant regulatory context
In order to define the most appropriate regulatory framework for the examined technologies, it is necessary to acknowledge the limitations of classifying them based on general social network or AI categories. While it is true that they belong to these macro-categories, what makes the examined cases distinctive is their ability to directly affect the users’ brain functions through addictive patterns.
This peculiar nature brings them close to neuro technologies, given that social networks and AI-powered chatbots have implemented product designs centred on human brain function and human-machine interaction, leveraging brain function to influence human behaviour in a targeted manner. While there is not yet a set definition of neuro technologies in the current early stage of the regulatory debate, UNESCO has emphasised the role of brain interfaces as a characterising element [UNESCO, 2025]. This underestimates the technological convergence between neuro technologies, AI, and digital environments for human-machine interaction (HMI), as well as the ability of AI-driven HMI to target brain functions in novel ways, inducing modifications without physical interaction.
This proximity to neuro technologies – which has led to the concept of ‘quasi-neurotechnologies’ [Mantelero, 2026] – and the serious concerns they have raised demand a greater degree of protection with regard to potentially affected rights. Primarily, this concerns the (largely unexplored) right to freedom of thought, considered broadly.
This right is relevant in the context of AI chatbots and social media because they can affect or manipulate users’ thoughts through interaction with users [Bublitz, 2026; Committee on the Rights of the Child, 2021]. It is important to note that the affected right is deemed an absolute right, which limits the acceptability of the potential risk of interference and makes associated AI-based solutions prohibited [Ligthart, 2025].
The right to mental integrity is also relevant, even though it is not absolute, as such technologies can alter the way the brain works (e.g. by impacting the effect of dopamine to induce addiction). Article 3(1) ECFR was intentionally inserted in response to technological interferences not adequately addressed at the bodily level, specifically those interfering with human brain functioning [Bublitz, 2024]. In this respect, it is important to consider what threshold makes these interferences relevant, given that many stimuli impact brain activity. Examining cases of addiction or of mental health conditions worsening to the point of death leaves no doubt that the design of the adopted AI technology has crossed this threshold.
Finally, and more broadly, human dignity encompasses caring for and protecting vulnerable individuals. This is compromised when people with mental health issues are encouraged by AI chatbots and social media to engage in harmful behaviour.
- Fundamental right protection and AI regulation
The AI Act does not introduce additional safeguards in terms of fundamental rights, nor does it create new rights or derogate from existing ones. Its contribution is to ensure that potential prejudice is reduced through a risk-based approach, specifically the assessment of the impact on fundamental rights, which is an obligation for both AI providers and deployers.
With respect to the aforementioned case, the AI Act therefore aligns with the previous considerations drawn from the general framework of fundamental rights. Article 5(1)(a) of the AI Act prohibits AI systems that deploy ‘purposefully manipulative or deceptive techniques, with the objective, or the effect of materially distorting the behaviour of a person or a group of persons by appreciably impairing their ability to make an informed decision, thereby causing them to take a decision that they would not have otherwise taken in a manner that causes or is reasonably likely to cause that person, another person or group of persons significant harm’ [European Commission, 2025; AI Act, Recital 29].
This prohibition covers the types of systems described briefly here [Ibid., para 88]. Article 5(1)(a) requires a significant harm, and the examined cases clearly demonstrate that this threshold is met. In this respect, given the nature of the potentially affected people – for example, those suffering from mental health problems or children – their vulnerable condition must also be considered [Ibid., para 92].
Prohibition under Article 5(1)(b) may apply to cases concerning social media when AI-powered services are designed for children based on their age, with the intention of exploiting their vulnerabilities, as in the discussed cases. The same applies to cases involving mental disabilities. Due to the potential partial overlap between cases under Article 5(1)(a) and (b), ‘the primary criterion for differentiation should be the dominant aspect of the exploitation’ [Ibid., paras 122-125].
Finally, in cases where the technology solutions are not addictive in the way recently experienced, nor impacting on the freedom of thought or human dignity, but affecting mental integrity or the right to health (mental health [WHO, 2023]), the risk assessment procedure – namely FRIA [Mantelero, 2024; APDCAT 2025; UNDP, 2025] – must be crucial in analysing the probability and severity of the risk, and in implementing the appropriate prevention and mitigation measures, including with regard to the systemic risk. The latter is particularly relevant in cases affecting young people and vulnerable groups on a large scale.
