UK (Parliament Politics Magazine) – The Children’s Commissioner for England urges an immediate ban on deepfake “nudification” apps, citing growing concerns over online abuse and exploitation.
As reported by The Guardian, AI-driven “nudification” apps that create deepfake sexual images of children should be banned, according to the Children’s Commissioner for England, amid fears of online abuse.
What did the report say about AI tools and children’s safety?
According to the children’s commissioner’s report, many girls have refrained from sharing photos online due to fears that generative AI tools could be used to remove their clothes or sexualize them. While it is illegal to create or distribute explicit images of children, the technology behind such deepfakes remains unregulated, the report states.
The report urges Ofcom to implement age checks on nudification apps and ensure social media platforms block the promotion of explicit deepfake content to children, under the Online Safety Act.
The report included a 2025 Girlguiding survey, which revealed that 26% of teenagers aged 13-18 had seen sexually explicit deepfake images of various individuals, including celebrities and teachers.
it highlighted the connection between deepfake abuse and mental health issues like suicidal thoughts and PTSD, referencing Mia Janin’s suicide in March 2021 as an example.
What did Dame Rachel de Souza say about the dangers of deepfake apps?
The commissioner, Dame Rachel de Souza, stated, “Children have told me they are frightened by the very idea of this technology even being available, let alone used. They fear that anyone – a stranger, a classmate, or even a friend – could use a smartphone as a way of manipulating them by creating a naked image using these bespoke apps.”
She said, “The online world is revolutionary and quickly evolving, but there is no positive reason for these particular apps to exist. They have no place in our society. Tools using deepfake technology to create naked images of children should not be legal and I’m calling on the government to take decisive action to ban them, instead of allowing them to go unchecked with extreme real-world consequences.”
The Children’s Commissioner urged the government to pass an AI bill requiring GenAI developers to address the risks of their products and create systems to remove explicit deepfakes of children, calling deepfake abuse a form of violence.
Ms De Souza added in the report that the new technology confronts children with unfamiliar concepts and is advancing so quickly that it’s hard to fully grasp its potential dangers.
What did the government say about AI-generated child abuse images?
A government spokesperson stated, “Creating, possessing or distributing child sexual abuse material, including AI-generated images, is abhorrent and illegal. Under the Online Safety Act platforms of all sizes now have to remove this kind of content, or they could face significant fines.”
They added, “The UK is the first country in the world to introduce further AI child sexual abuse offences, making it illegal to possess, create or distribute AI tools designed to generate heinous child sexual abuse material.”
What did lawyers say about the impact of deepfake technology on teenage boys and its legal consequences?
According to lawyers, teenage boys are increasingly facing arrest for sexual offences due to a lack of awareness. Cases include involvement with deepfakes, explicit WhatsApp chats, and viewing pornography featuring peers.
Danielle Reece-Greenhalgh, a partner at Corker Binning, specializes in sexual offences and indecent images. She stated that the law is failing to keep up with the surge in deepfake technology. This issue is creating major problems for law enforcement in protecting victims of abuse.
She pointed out that banning these apps might fuel discussions on internet rights and have an unequal impact on young men, who often use AI without understanding the consequences.
Ms Reece-Greenhalgh explained that while the criminal justice system seeks to avoid punishing typical teenage behaviour, arrests can lead to serious consequences. These consequences may include difficulties at school or issues with background checks.
Matt Hardcastle from Kingsley Napley described the online environment as a “minefield” for young people, where accessing illegal sexual and violent material is alarmingly easy. He stressed that many parents don’t realize how quickly their children can be led into a dark space, such as through nudification apps.
He said, “They’re looking at it through the eyes of a child. They’re not able to see that what they’re doing is potentially illegal, as well as quite harmful to you and other people as well,” adding, “Children’s brains are still developing. They have a completely different approach to risk-taking.”
Marcus Johnstone, a criminal solicitor, said, “Often parents had no idea what was going on. They’re usually young men, very rarely young females, locked away in their bedrooms and their parents think they’re gaming. These offences didn’t exist before the internet, now most sex crimes are committed online. It’s created a forum for children to become criminals.”
What did Peter Kyle say about the wind-down feature and online safety for children?
Technology Secretary Peter Kyle said he was “watching very carefully” the effects of the wind-down feature.
He said, “These are things I am looking at. I’m not going to act on something that will have a profound impact on every single child in the country without making sure that the evidence supports it – but I am investing in [researching] the evidence.”
Mr Keyle added, “Growing up in the digital age should mean children can reap the immense benefits of the online world safely, but in recent years too many young people have been exposed to lawless, poisonous environments online which we know can lead to real and sometimes fatal consequences. This cannot continue.”