Introduction
Since the early internet, the production and dissemination of child sexual abuse material (CSAM) was identified as the paradigmatic case for targeting illegal content online, and platforms have long voluntarily cooperated in deploying technological mechanisms for detection and removal. What generative artificial intelligence has fundamentally changed is the character of the problem: abusive material is now scalable and accessible without any physical contact with a child; the boundary between real and entirely AI generated victims has blurred; and the legal question has shifted from the individual image to the technical infrastructure that enables mass production and circulation. Visual content can be generated using open-source AI models with few or no safety guardrails, as well as dedicated applications available on ordinary app stores. This is not only a dark-web phenomenon. It is an infrastructural one. The Center for Countering Digital Hate reported that Grok, the generative AI tool integrated into X, generated around 23,000 sexualised images of children over an eleven-day period (The Center for Countering Digital Hate, 2026). At the same time, AI-based detection tools are themselves being deployed to counter AI-generated CSAM — a dynamic in which the same generative AI both creates harm and is pressed into the service of its prevention, raising urgent questions about technical reliability, false positives, and the accountability of automated enforcement.
How is EU law responding to the challenges posed by AI-generated CSAM? This contribution briefly examines three distinct but interconnected layers: section 1 addresses the criminal law framework and its inherent limits as a tool of prevention; section 2 analyses the horizontal digital regulation layer — the AI Act and the Digital Services Act; section 3 examines the proposed sectoral CSAM Regulation.
1. Criminal Law Dimension
The criminal-law foundation is Directive 2011/93/EU on combating the sexual abuse and sexual exploitation of children, based on Articles 82 and 83 TFEU, establishing minimum rules on offences and sanctions. Article 2 already covered «realistic images of a child engaged in sexually explicit conduct» and material depicting «any person appearing to be a child», concepts that anticipated, at least in part, the generative AI challenge.
The proposed recast published by the Commission in February 2024 (COM/2024/60 final) substantially modernises the framework by introducing new technological offences. It replaces the terminology of «child pornography» with the more comprehensive CSAM and extends the definition to cover not only realistic images but expressly «reproductions and representations» of children engaged in sexually explicit conduct, designed to capture deepfakes, avatars, synthetic bodies, sensory feedback and immersive environments. The Commission justifies this extension as a form of «offender-side prevention»: accessing abusive material is characterised as «often the first step towards hands-on abuse, regardless of whether it depicts real or simply realistic abuse and exploitation» (recital 11). Pedophile manuals — material providing advice on how to find, groom and abuse children while avoiding detection — are likewise brought within the CSAM definition (recital 12). This extension matters not only for criminal liability but also because it directly feeds the scope of private enforcement obligations: the broader the criminal definition, the wider the category of material that platforms are expected to detect, remove and report. The provisional agreement reached between Parliament and Council in June 2026 on the recast Directive (EP press release, 22 June 2026) pushes the criminal framework further. It criminalises AI systems «designed or adapted specifically to produce» CSAM, making acquiring, possessing or disseminating such systems punishable. Criminal law extensions raise a question well-known to comparative criminal law scholars (Caletti, 2024): how far can preventive criminal law expand before it loses its character as a response to concrete harm when no real child is depicted and becomes, simultaneously, an instrument of moral governance and the normative basis for sweeping private enforcement obligations? A further complexity concerns the peer-to-peer dimension. Criminological analysis reveals that the profile of those producing AI-generated CSAM is heterogeneous: it includes not only pedophile offenders but also adolescents engaged in peer harassment using tools available on ordinary app stores. A 2025 survey of 1,200 young people aged 13 to 20 found that one in ten teenagers personally knew someone targeted by deepfake nude imagery, and one in seventeen disclosed being a direct victim (Thorn, 2025). The proposed recast attempts to address this (Rigotti et al., 2025), by clarifying that consensual sexual activities among peers defined as «persons close in age in their psychological and physical development or maturity should not always be punished».
2. Digital Regulation
EU law does not respond to AI-generated CSAM through criminal law alone. Private platforms occupy a key position in this response they control the infrastructure through which CSAM is produced, circulated. The response is accordingly multi-layered, combining the AI Act (Regulation (EU) 2024/1689), the DSA (Regulation (EU) 2022/2065), as well as the proposed CSAM Regulation (COM (2022) 209 final). Each instrument operates at a different level: digital regulation instruments operate horizontally, while the proposed CSAM Regulation is a sectoral instrument, specifically targeting child sexual abuse. They all rely on Article 114 TFEU as their legal basis for internal market harmonisation.
The AI Act intervenes at the level of system design and deployment. Article 50 imposes transparency obligations for certain AI-generated or manipulated content. More significantly, the AI Digital Omnibus (COM/2025/836) has added a new prohibited practice under Article 5 of the AI Act, covering AI systems designed to generate non-consensual sexually explicit content or CSAM — a prohibition that will exclude ‘nudification’ tools and similar applications from the EU market entirely. This represents a decisive shift: from risk management and disclosure towards outright prohibition of certain AI functionalities.
The DSA provides a procedural and systemic risks layer. It treats CSAM as illegal content, imposing notice-and-action obligations, trusted-flagger mechanisms, and for very large online platforms and search engines risk assessment and mitigation duties regarding the protection of minors. The Commission’s 2025 Guidelines under Article 28(4) DSA, specifically directed at very large platforms, reinforce these obligations by requiring platforms to address, by design, risks arising from AI chatbots and companions, recommender systems and AI-driven deepfakes, and to implement default privacy settings, reporting tools accessible to children, and safeguards against unwanted contact. On 26 January 2026, the Commission opened a formal investigation into Grok on X under the DSA to assess whether systemic risks linked to AI-generated CSAM had been adequately mitigated, a direct application of the systemic risk governance framework to AI-generated abusive material (King and Portante D’Alessandro, 2026).
Together, the AI Act and DSA create an upstream dimension that criminal law cannot provide on its own: they govern the design and operation of systems before harm occurs and hold platforms accountable for the architecture through which CSAM is produced, amplified and circulated.
3. The way forward: the proposed CSA Regulation
The proposed CSA Regulation, widely known as «Chat Control» (Jakubowska, 2025), is the most contested element of the EU framework, and the one where the structural tensions between child protection, privacy, and the delegation of public enforcement to private actors are sharpest.
The Commission proposed this as a permanent framework to replace the temporary derogation from the ePrivacy Directive (Directive 2002/58/EC) granted by Regulation (EU) 2021/1232, which allowed providers of number-independent interpersonal communications services (NI-ICS) — messaging and webmail services — to voluntarily detect, report and remove CSAM without breaching ePrivacy confidentiality obligations. Lately, the Council and Parliament reached agreement to reinstate the interim framework until 3 April 2028, explicitly framing the measure as a bridge «until a new EU law offers a long-term legal framework» — without prejudice to the ongoing negotiations on the permanent Regulation.
Despite its internal market legal basis, the proposed Regulation clearly serves security purposes and expressly seeks to enhance the criminal law framework of the Child Sexual Abuse Directive. It builds on the DSA’s horizontal architecture but goes further, imposing more specific preventive obligations on certain providers. In particular, it pursues three objectives: (i) mandatory risk assessments to identify whether services could be misused for the dissemination of CSAM or for grooming, and corresponding mitigation obligations; (ii) detection orders issued by judicial or independent administrative authorities, requiring providers to scan for known CSAM (via hash-matching databases), new CSAM — including AI-generated synthetic material — and grooming-related content; and (iii) the creation of an EU Centre on child sexual abuse, as a decentralized EU body coordinating with Europol and national law enforcement, verifying provider reports, and supplying detection technologies. A central role is hence played by the several kinds of technological solutions service providers may potentially use to fulfil their obligations and execute detection orders. The 2023 complementary Impact Assessment by the European Parliamentary Research Service (EPRS, 2023) concluded that existing technologies for detecting new CSAM and grooming exhibit high false-positive rates, are susceptible to evasion, and struggle with context-dependence; the EDPS and EDPB emphasised the same concerns in their Joint Opinion 04/2022. The original proposal applied to all services, including end-to-end encrypted ones, a requirement that critics argued effectively mandates client-side scanning, functionally equivalent to undermining encryption and amounting to a general monitoring obligation incompatible with the Charter of Fundamental Rights. The legislative process has been protracted for a long. The European Parliament’s November 2023 mandate significantly narrowed the scope, excluding grooming and text-based communications from detection orders and insisting on judicial authorisation as a last resort. On 26 November 2025, the Council adopted its General Approach, removing mandatory detection orders and retaining only DSA-aligned risk assessment obligations, with voluntary scanning as the main residual practice. Trilogue negotiations began in December 2025. The pattern that emerges is characteristic of what (ten Hulsen, 2025) defines as techno-solutionism: a complex societal problem is reframed as one solvable with the right technology in place, assuming a level of technical reliability and social precision that available tools do not possess. Simeon de Brouwer of EDRi noted that the approach amounts to «outsourcing to private companies the work of law enforcement without any safeguards» — a blank cheque for technology companies to scan private communications and report to a US-based centre before onward transmission to EU law enforcement (Euronews, 18 July 2026). On the other side, the Internet Watch Foundation has equally warned that without mandatory detection obligations, the EU’s capacity to identify victims will diminish, a concern that should not be dismissed (IWF, 2026).
Brief conclusion
Protecting children in the digital environment is a priority, and the response must certainly be multi-layered, firstly by addressing the structural conditions that make the online environment harmful. Generative AI has expanded both the notion and the infrastructure of CSAM, and is now simultaneously pressed into the service of its own prevention. This double function — creator of harm and instrument of detection — makes AI-generated CSAM a current test of the EU model of criminal law, platform regulation and fundamental rights. There are three main risks: an expanding perimeter of preventive criminal law; a model of private detection – whether voluntary or ordered – which systematically delegates public law enforcement functions to private actors without the safeguards, accountability mechanisms or legitimacy that public enforcement carries; and a techno-solutionist faith in detection technologies that are not technically capable of delivering what the political discourse promises, including chilling effects on victim reporting.
