Inside the deepfake porn crisis engulfing Korean schools

Last Saturday, a Telegram message popped up on Heejin’s phone from an anonymous sender. “Your pictures and personal information have been leaked. Let’s discuss.”

As the university student entered the chatroom to read the message, she received a photo of herself taken a few years ago while she was still at school. It was followed by a second image using the same photo, only this one was sexually explicit, and fake.

Terrified, Heejin, which is not her real name, did not respond, but the images kept coming. In all of them, her face had been attached to a body engaged in a sex act, using sophisticated deepfake technology.

Deepfakes, the majority of which combine a real person’s face with a fake, sexually explicit body, are increasingly being generated using artificial intelligence.

“I was petrified, I felt so alone,” Heejin told the BBC.

But she was not alone.

Two days earlier, South Korean journalist Ko Narin had published what would turn into the biggest scoop of her career. It had recently emerged that police were investigating deepfake porn rings at two of the county’s major universities, and Ms Ko was convinced there must be more.

She started searching social media and uncovered dozens of chat groups on the messaging app Telegram where users were sharing photos of women they knew and using AI software to convert them into fake pornographic images within seconds.

“Every minute people were uploading photos of girls they knew and asking them to be turned into deepfakes,” Ms Ko told us.

Ms Ko discovered these groups were not just targeting university students. There were rooms dedicated to specific high schools and even middle schools. If a lot of content was created using images of a particular student, she might even be given her own room. Broadly labelled “humiliation rooms” or “friend of friend rooms”, they often come with strict entry terms.

Ms Ko’s report in the Hankyoreh newspaper has shocked South Korea. On Monday, police announced they were considering opening an investigation into Telegram, following the lead of authorities in France, who recently charged Telegram’s Russian founder for crimes relating to the app. The government has vowed to bring in stricter punishments for those involved, and the president has called for young men to be better educated.

Telegram said it "actively combats harmful content on its platform, including illegal pornography," in a statement provided to the BBC.

'A systematic and organised process'

The BBC has viewed the descriptions of a number of these chatrooms. One calls for members to post more than four photos of someone along with their name, age and the area they live in.

“I was shocked at how systematic and organised the process was,” said Ms Ko. “The most horrific thing I discovered was a group for underage pupils at one school that had more than 2,000 members.”

In the days after Ms Ko’s article was published, women’s rights activists started to scour Telegram too, and follow leads.

By the end of that week, more than 500 schools and universities had been identified as targets. The actual number impacted is still to be established, but many are believed to be aged under 16, which is South Korea's age of consent. A large proportion of the suspected perpetrators are teenagers themselves.

Heejin said learning about the scale of the crisis had made her anxiety worse, as she now worried how many people might have viewed her deepfakes. Initially she blamed herself. “I couldn’t stop thinking did this happen because I uploaded my photos to social media, should I have been more careful?”

Scores of women and teenagers across the country have since removed their photos from social media or deactivated their accounts altogether, frightened they could be exploited next.

“We are frustrated and angry that we are having to censor our behaviour and our use of social media when we have done nothing wrong,” said one university student, Ah-eun, whose peers have been targeted.

Ah-eun said one victim at her university was told by police not to bother pursuing her case as it would be too difficult to catch the perpetrator, and it was “not really a crime” as “the photos were fake”.

News 1South Koreans have been shocked and angry since the deepfake scandal has come to light

At the heart of this scandal is the messaging app Telegram. Unlike public websites, which the authorities can access easily, and then request for images be removed, Telegram is a private, encrypted messaging app.

Users are often anonymous, rooms can be set to “secret” mode, and their contents quickly deleted without a trace. This has made it a prime space for criminal behaviour to flourish.

Last week, politicians and the police responded forcefully, promising to investigate these crimes and bring the perpetrators to justice.

On Monday, Seoul National Police Agency announced it would look to investigate Telegram over its role in enabling fake pornographic images of children to be distributed.

The app’s founder, Pavel Durov, was charged in France last week with being complicit in a number of crimes related to the app, including enabling the sharing of child pornography.

But women's rights activists accuse the authorities in South Korea of allowing sexual abuse on Telegram to simmer unchecked for too long, because Korea has faced this crisis before. In 2019, it emerged that a sex ring was using Telegram to coerce women and children into creating and sharing sexually explicit images of themselves.

Police at the time asked Telegram for help with their investigation, but the app ignored all seven of their requests. Although the ringleader was eventually sentenced to more than 40 years in jail, no action was taken against the platform, because of fears around censorship.

“They sentenced the main actors but otherwise neglected the situation, and I think this has exacerbated the situation,” said Ms Ko.