🔗 Share this article

Tech firms and child safety organizations will receive permission to assess whether artificial intelligence systems can produce child exploitation images under new British legislation.

Significant Increase in AI-Generated Harmful Material

The declaration came as revelations from a protection monitoring body showing that reports of AI-generated child sexual abuse material have increased dramatically in the last twelve months, growing from 199 in 2024 to 426 in 2025.

New Regulatory Structure

Under the amendments, the government will allow designated AI developers and child safety organizations to inspect AI models – the foundational systems for conversational AI and image generators – and verify they have sufficient protective measures to prevent them from creating images of child exploitation.

"Ultimately about stopping abuse before it happens," stated Kanishka Narayan, noting: "Specialists, under strict protocols, can now detect the risk in AI systems early."

Tackling Legal Challenges

The changes have been introduced because it is illegal to create and possess CSAM, meaning that AI developers and other parties cannot generate such content as part of a testing regime. Until now, authorities had to wait until AI-generated CSAM was uploaded online before dealing with it.

This law is designed to preventing that problem by helping to stop the creation of those images at their origin.

Legal Framework

The changes are being introduced by the authorities as revisions to the criminal justice legislation, which is also implementing a ban on owning, producing or sharing AI models developed to create exploitative content.

Practical Impact

This recently, the official toured the London headquarters of Childline and listened to a mock-up conversation to advisors featuring a report of AI-based abuse. The call depicted a adolescent seeking help after being blackmailed using a sexualised deepfake of himself, created using AI.

"When I hear about children facing extortion online, it is a cause of intense anger in me and justified concern amongst families," he stated.

Alarming Statistics

A prominent online safety foundation reported that cases of AI-generated abuse material – such as webpages that may include numerous files – had more than doubled so far this year.

Instances of category A content – the most serious form of exploitation – rose from 2,621 visual files to 3,086.

• Girls were overwhelmingly targeted, making up 94% of illegal AI depictions in 2025
• Depictions of infants to toddlers rose from five in 2024 to 92 in 2025

Industry Response

The legislative amendment could "constitute a vital step to ensure AI tools are secure before they are released," stated the head of the online safety organization.

"AI tools have made it so victims can be victimised repeatedly with just a few clicks, providing criminals the ability to make possibly endless quantities of sophisticated, photorealistic exploitative content," she added. "Content which further commodifies victims' suffering, and makes children, especially female children, more vulnerable on and off line."

Counseling Interaction Information

Childline also released details of counselling interactions where AI has been referenced. AI-related risks mentioned in the sessions include:

• Employing AI to rate body size, body and appearance
• Chatbots dissuading children from consulting trusted adults about abuse
• Being bullied online with AI-generated content
• Digital extortion using AI-faked pictures

Between April and September this year, Childline delivered 367 counselling sessions where AI, conversational AI and related topics were discussed, four times as many as in the equivalent timeframe last year.

Half of the mentions of AI in the 2025 sessions were connected with mental health and wellness, encompassing utilizing chatbots for assistance and AI therapy applications.