- A Toy Story: The Rise of AI-Powered Toys and Fundamental Rights Concerns
The recently released Toy Story 5 depicts the iconic characters Woody, Jessie and Buzz confronting the arrival of a technologically advanced new toy, Lilypad. As their owner, Bonnie, becomes increasingly attached to an interactive tablet, the traditional toys try to show the device that genuine social interaction and friendship are far more valuable to the little girl than the digital companionship it provides through its screen.
Far from being a purely fictional scenario, the rise of so-called smart and AI-powered toys is already a reality. Internet of Things (IoT) technologies and AI-enabled tools are becoming increasingly common in our lives, and the toy industry is no exception.
Smart toys, generally belonging to the IoT sphere, connect via Wi-Fi or Bluetooth and integrate microphones, cameras, sensors and other connectivity features to collect and transmit data. A new generation of toys, however, goes a step further by incorporating AI systems, including chatbots and AI companions. AI-powered toys can thus interact with children, remember previous conversations, personalise future interactions and engage in real-time dialogue. By simulating empathy and adapting to users over time, including through chatbot functionalities, these toys create a sense of care and emotional attachment, encouraging increasingly prolonged interaction.
As the global market for AI toys continues to grow, unprecedented concerns have emerged. Unlike traditional toys, whose main dangers were related to physical safety, AI-powered toys raise far broader risks. While marketed as tools to foster creativity, learning and social skills, this new generation of toys, much like AI companions more generally, gives rise to perils that include not only privacy and data protection but also potential long-term psychological effects resulting from addictive design, manipulation and behavioural nudging. Such technologies may influence children’s emotional and social development as well as mental health, particularly given kids’ heightened vulnerability and their limited ability to distinguish AI agents from human relationships or exercise fully developed critical judgment.
Based on these concerns, several smart and AI-powered toys have been the subject of advocacy campaigns and, in some cases, have even been withdrawn from the market. Examples include My Friend Cayla, which was banned in Germany in 2017 for violating telecommunications law, as its built-in microphone could transmit conversations without users being sufficiently aware; but also Kumma, an AI teddy bear, whose service was suspended after reports of sexually explicit terms appearing in interactions with children. These cases illustrate how AI toys may involve pervasive listening, the collection and processing of personal data – including biometric data –, cybersecurity vulnerabilities, opaque data practices, the profiling of children and broader menaces of manipulation and surveillance.
Given the well-established principle according to which all regulatory approaches concerning children must prioritize their best interests, as recognised by Article 24(2) of the EU Charter of Fundamental Rights and Article 3 of the UN Convention on the Rights of the Child, the emergence of such profound and unprecedented risks to children’s fundamental rights has prompted a regulatory debate on how to ensure their effective protection in an increasingly digitalised and automated playground. This blog post examines the current legislative landscape in the European Union (EU) and the United States (US), highlighting persistent gaps and challenges.
- The EU Regulatory Landscape: From Data Protection and Cybersecurity to AI-Specific Discipline, Passing through the Safety of Toy Regulation
The EU regulatory framework on toys has witnessed significant developments in recent years. While the GDPR (Reg. (EU) 2016/679) remains applicable to any processing of personal data collected by smart and AI-powered toys, the emergence of interconnected toys has also been addressed by the Cyber Resilience Act (Reg. (EU) 2024/2847). This legislation recognises the need for reinforced safeguards for digital toys, as they may affect vulnerable consumers (Recital 10). For this reason, such toys are classified as Class I “important products with digital elements” (Annex III), thus requiring enhanced guarantees and conformity assessment procedures (Art. 7).
The AI Act (Reg. (EU) 2024/1689) also applies to AI-powered toys. While the particular vulnerability of children is acknowledged in Recitals 28 and 48, Recital 50 clarifies that AI systems which are safety components of products, or which are themselves products falling within the scope of Union harmonisation legislation listed in Annex I, are classified as high-risk when the product is subject to a third-party conformity assessment under the relevant sectoral legislation. The Directive 2009/48/EC on the safety of toys (subsequently replaced by Reg. (EU) 2025/2509, as will be discussed) is explicitly included in Annex I. Falling under the definition of high-risk systems, AI toys are consequently subject to extensive requirements on conformity assessment, risk management, transparency, safety-by-design and other related obligations (Art. 6 and following).
Moreover, where AI systems embedded in toys are capable of “exploiting vulnerabilities of natural persons or specific groups of persons due to their age, disability, or specific social or economic situation, with the objective or effect of materially distorting their behaviour in a manner that causes or is reasonably likely to cause significant harm”, they have to be classified as systems presenting an unacceptable risk under Article 5(1)(b) and are therefore prohibited.
The AI Act has nonetheless undergone recent amendments, also affecting provisions relevant to toys. Following the agreement on the so-called “Digital Omnibus on AI”, reached in May 2026, and the final approval by the Council on 29 June, the application of high-risk provisions has been postponed. The revised timeline sets application dates at 2 December 2027 for stand-alone high-risk AI systems and 2 August 2027 for high-risk AI systems embedded in products, such as toys. In addition, the Omnibus amendment addresses a sensitive issue: the interaction between AI rules and sectoral legislation in areas such as medical devices, toys, lifts and watercraft. The final purpose is that of reducing overlaps and administrative burdens for companies required to comply with parallel sets of obligations. To this end, the Omnibus attributes to the Commission the possibility to adopt delegated acts specifying which AI Act requirements should be limited in cases where sectoral legislation contains equivalent AI-specific provisions, in order to facilitate compliance and minimise duplication “while preserving the level of protection ensured by that Regulation” (Recital 5). It remains to be seen whether the Commission will adopt implementing acts concerning AI toys specifically, in order to coordinate compliance pathways with sectoral toy safety law.
This coordination will become even more important in light of the aforementioned Safety of Toys Regulation (Reg. (EU) 2025/2509), which expressly recalls and cross-refers to the AI Act for specific requirements while also introducing additional safeguards (with full mandatory application scheduled for 1 August 2030). Although this Regulation does not directly discipline AI toys in their entirety, it acknowledges that “digital technologies have led to new hazards in toys” (Recital 14) and reinforces the need to comply with privacy rules, cybersecurity frameworks and the AI Act. It further confirms that toys incorporating AI systems as safety components requiring third-party conformity assessment are classified as high-risk AI systems under Article 6(1) of the AI Act (Recital 15). Beyond cross-references to other legislation, the Regulation introduces an important novelty. Recital 16 states that “the safety assessment should take into account the health risk posed by digitally connected toys, where appropriate, including any risk posed to mental health. Therefore, when assessing the safety of digitally connected toys likely to have an impact on children, manufacturers should ensure that the toys they make available on the market meet the highest standards of safety, security and privacy by design, in the best interests of children”. This approach is confirmed in Article 25(2)(c), which requires that, for toys falling within the scope of Regulations (EU) 2024/1689 or (EU) 2024/2847 or Directive 2014/53/EU, manufacturers must take into account the particular vulnerabilities of children when assessing and addressing the risks that the toy may present.
- A Fragmented Landscape in the United States: Between Sectoral Regulation and Emerging Moratoria
Looking to the other side of the Atlantic, the debate on AI-powered toys is also gaining momentum in the US. As is well known, at the federal level the US approach to AI regulation and governance – particularly following the most recent Presidential Executive Orders – has led to the absence of a comprehensive AI-specific regulatory framework. No targeted rules have been adopted either to address AI-powered toys. Nevertheless, several existing legal frameworks may apply to these products.
The Children’s Online Privacy Protection Act of 1998 (COPPA), enforced by the Federal Trade Commission (FTC), establishes rules governing online services directed at children, including requirements relating to parental consent, data security and the processing of children’s personal information. In response to the growing diffusion of connected toys, the FTC recently updated its COPPA Business Guidance, explicitly clarifying that connected toys fall within COPPA’s scope. On this basis, the FTC has brought several enforcement actions against toy manufacturers for violations of data protection and parental consent requirements. These interventions, nonetheless, have been confined to privacy and data protection issues. More recently, the FTC also launched an inquiry into AI chatbots acting as companions, seeking information on how companies “measure, test, and monitor potentially negative impacts of this technology on children and teens”.
Additional legal frameworks apply to the physical safety of toys, notably through the Consumer Product Safety Commission (CPSC), which establishes product safety and cybersecurity standards, as well as through general consumer protection law. However, these legislative frameworks largely predate the emergence of AI and are not well equipped to address the broader risks posed by AI-enabled toys and conversational agents, including potential long-term effects on children’s health, emotional well-being and social development. These delicate and serious issues do not appear to be fully captured by the existing regulatory framework.
Recognising these regulatory gaps, several legislative initiatives have recently been introduced at federal as well as state levels. At the federal level, the legislative focus has been on protecting children online, particularly on social media platforms. These include the Kids Online Safety Act – KOSA (S.1748) and the Children and Teens’ Online Privacy Protection Act, commonly referred to as COPPA 2.0 (S.836), both of which primarily aim to strengthen privacy and data protections. Other proposals address AI more broadly and could therefore have indirect implications also on AI toys. These include the AI LEAD Act (S.2937), a bipartisan proposal intended to classify AI systems as “products” and create a federal cause of action for product liability claims arising from AI-related harm, and the GUARD Act (S.3062), which would require AI chatbots to implement age-verification measures and provide specific disclosures. All of these proposals are not specifically targeted at AI-enabled toys and remain at an early stage of the legislative process.
At the state level, by contrast, some states are discussing legislation explicitly addressing AI toys. While numerous broader AI regulatory proposals could also apply to AI-powered toys – i.e. initiatives under discussion in Oregon and Washington – a recent proposal introduced in California, SB-867, would establish a temporary ban on AI-powered companion chatbot toys until 1 January 2031. The bill has passed the Senate and is currently under scrutiny by the Assembly. Similarly, in the State of New York, Bill S.9408A/A11144B aims to introduce a five-year moratorium on the sale of chatbot toys intended for young children, while an interagency task force should conduct a comprehensive assessment of the risks posed by such products. The bill has passed both the Senate and the Assembly.
Maryland has proposed the introduction of the Artificial Intelligence Toy Safety Act (HB1261), which would represent a dedicated regulatory framework for AI-enabled children’s toys. The proposal would require enhanced privacy protections, child safety assessments, parental consent mechanisms and the creation of an oversight body. The bill is currently under consideration in the House.
- From Toy Story to Children’s Story: A Necessary Shift in Perspective
While the current political debate remains mainly focused on social media for minors, the spread of AI toys is also becoming increasingly prominent.
In the EU, the discourse appears more mature, with recent regulations recognising and incorporating specific safeguards concerning AI toys. Not only does the AI Act ensure a comprehensive approach, addressing different risks to fundamental rights, but the recently adopted Regulation on the safety of toys also introduces the need for specific assessment of the health risks posed by connected and digital toys. Nevertheless, several – and potentially – overlapping legislative frameworks are currently in place: coordination and interplay between them may therefore prove challenging. The “Digital Omnibus on AI” initiative seeks to improve compliance and coordination and to create a more integrated framework. However, it remains to be seen whether the Commission will intervene through delegated acts in the toy sector, assessing whether sectoral legislation contains AI-specific requirements and whether such requirements are equivalent to those of the AI Act. Moreover, the implementation of many of the abovementioned rules has been postponed, so their actual effectiveness still needs to be assessed. It also remains to be seen if AI toys, especially those incorporating chatbots, will be considered capable of manipulation or of promoting dangerous behaviours, and will therefore be prohibited under the unacceptable risk provisions of the AI Act.
In the US, the absence of comprehensive federal rules on AI as well as on AI toys has not prevented legislative proposals from emerging mainly at state level – although, under the President Trump administration, the general direction is to avoid fragmented state-level regulation. At the federal level, existing legislations applicable to toys need to be complemented by rules capable of addressing the variety and complexity of risks posed by AI-powered products; this is clearly reflected in the statement that current authorities, such as the CPSC, are not “authorised to evaluate non-physical hazards, such as mental, emotional, or psychological harm, or physical harm that is not proximately caused by a product’s physical characteristics or operation”. Several state-level proposals, by contrast, recognise the need to take such risks into account; the uncertainties related to these concerns have therefore led to specific proposals including ad hoc regulations or general bans, inspired by a precautionary approach, particularly with regard to AI chatbots embedded in toys. While moratoria are temporary solutions, they clearly reflect the need to further stimulate the debate on these technologies and develop regulatory frameworks capable of ensuring children’s rights.
These recent legislative activities signal growing attention to safety, transparency and the appropriate use of AI-driven features in consumer products designed to interact directly with minors. Nonetheless, there is a need to establish comprehensive rules and promote a holistic approach: children’s vulnerability should be carefully integrated into legislative choices as well as risk assessments. Legislators, authorities and businesses should consequently be required to address not only physical dangers but also non-physical harms, such as psychological risks or impacts on the development of minors.
As Buzz states in the initially recalled Toy Story 5 movie, a toy’s “mission on this planet is to make a child happy”. Legislators, policymakers, public authorities, but also companies and civil society, should ensure that this mission is preserved also in the digital era.
