The creator of sexualised pictures of pre-teen girls says he uses open-source AI image-maker Stable Diffusion
By Joe Tidy
Cyber correspondent
A leading children’s charity is calling on Prime Minister Rishi Sunak to tackle AI-generated child sexual abuse imagery, when the UK hosts the first global summit on AI safety this autumn.
The Internet Watch Foundation (IWF) removes abuse content from the web and says AI images are on the rise.
Last month, the IWF began logging AI images for the first time.
It discovered predators around the world sharing galleries of sometimes photo-realistic pictures.
“We are not currently seeing these images in huge numbers, but it is clear to us the potential exists for criminals to produce unprecedented quantities of lifelike child sexual abuse imagery,” said Susie Hargreaves, the IWF’s chief executive.
The BBC was shown redacted versions of some of the images, which showed girls about five years old posing naked in sexual positions.
The IWF is one of only three charities in the world licensed to actively search for child abuse content online.
It began logging AI images on 24 May and says that by 30 June analysts had carried out investigations on 29 sites, and confirmed seven pages sharing galleries containing AI images.
The charity did not confirm the exact number of images, but says dozens of AI pictures were mingled with real abuse material being shared on the illegal sites.
Some of them were what the experts class as Category A images – the most graphic possible, depicting penetration.
It is illegal to create images of child sexual abuse in almost every country.
“We have a chance, now, to get ahead of this emerging technology, but legislation needs to be taking this into account, and must be fit for purpose in the light of this new threat,” said Ms Hargreaves.
In June, Mr Sunak announced plans for the UK to host the world’s first global summit on AI safety.
The government promises to bring together experts and lawmakers to consider the risks of AI and discuss how they can be mitigated through internationally co-ordinated action.
As part of their work, IWF analysts record trends in abuse imagery, such as the recent rise of so-called “self-generated” abuse content, where a child is coerced into sending videos or images of themselves to predators.
The charity is concerned that AI-generated imagery is a growing trend, although the number of discovered images is still a fraction of other forms of abuse content.
In 2022, the IWF logged and attempted to take offline more than 250,000 web pages containing child sexual abuse imagery.
Analysts also recorded conversations on forums between predators, who shared tips on how to create the most lifelike images of children.
They found guides on how to trick AI into drawing abuse images, and how to download open-source AI models and remove the safety barriers.
Whilst most AI image generators have strict built-in rules to stop users generating content with banned words or phrases, open-source tools can be downloaded for free and tweaked however the user wants.
The most popular source is Stable Diffusion. Its code was released online in August 2022 by a team of German AI academics.
The BBC spoke to an AI image-maker who uses a specialised version of Stable Diffusion to create sexualised images of pre-teen girls.
The Japanese man claimed that his “cute” images were justified, and said it was “the first time in history that images of children can be made without exploiting real children”.
However, experts say the images have the potential to cause serious harm.
“There’s no doubt in my mind that AI-generated images are going to increase these predilections, reinforce this deviance, and it’s going to lead to greater harm and greater risk of harm to children around the world,” said Dr Michael Bourke, who specialised in sex offenders and paedophiles for the United States Marshals Service.
Prof Bjorn Ommer, one of Stable Diffusion’s lead developers, defended the decision to make it open source. He told the BBC hundreds of academic research projects have since been borne out of it, including many thriving businesses.
Prof Ommer suggests this is vindication for his and his team’s decision, and insists stopping research or development is not the right thing to do.
Prof Bjorn Ommer is one of five authors of the Stable Diffusion AI model
“We really need to face the fact that this is a worldwide, global development. Stopping it here would not stop the development of this technology globally in other countries that are probably non-democratic societies. We need to figure out mitigation steps to consider this development we are seeing moving fast,” he said.
Stability AI, which helped fund the development of the model pre-launch, is one of the most prominent companies building new versions of Stable Diffusion. It declined to be interviewed, but has previously said it prohibits any misuse of its versions of the AI for illegal or immoral purposes.
UK viewers can see more on Newsnight on BBC Two at 22:30 BST on Monday or on BBC iPlayer afterwards.
