The government has announced plans to collaborate with tech firms, including Microsoft, and other partners to develop tools to combat the unprecedented growth of deepfakes.

The aim is to mitigate the proliferation of harmful AI-generated deepfake content, which the government says is now an “urgent national priority”.

While deepfakes have existed for almost a decade, rapid advances in generative AI in recent years have driven their scale and sophistication across the internet.

As its name suggests, AI technology is used to create images and videos that appear highly realistic but are in fact fake. The ability to create convincing deepfakes quickly and cheaply is available to almost anyone, with little to no technical expertise required.

According to recent government figures, the rise in AI-generated deepfakes has grown from 500,000 shared in 2023 to a projected eight million in 2025. 

The government has already banned ‘nudification’ apps, which use generative AI to turn images of real people into deepfake nude pictures and videos without their consent. 

In addition to this legislation, it is now also exploring ways to quickly detect and mitigate harmful deepfake content. In what it calls a “world-first deepfake detection initiative”, the government will work with tech firms, including Microsoft, as well as academics and other experts to set standards for tackling harmful and deceptive AI-generated content.

The framework will evaluate how detection technologies can identify, assess and understand harmful deepfake content, regardless of its source. These tools will be tested against real-world threats such as sexual abuse, fraud and impersonation. The aim is to also provide a better understanding of where technological limitations remain.

Jess Phillips, minister for safeguarding, said the framework would address what she called an “injustice affecting millions”. 

"For the first time, this framework will take the injustice faced by millions to seek out the tactics of vile criminals, and close loopholes to stop them in their tracks so they have nowhere to hide,” she said.

The framework builds on work undertaken by the Accelerated Capability Environment (ACE), a Home Office-initiated public-private partnership that aims to solve rapidly-changing digital challenges. In 2024, ACE ran a Deepfake Detection Challenge that resulted in six teams – including IBM, Oxford Wave Research and Open Origins – developing solutions now undergoing benchmark testing and user trials. 

Last year, a study conducted by researchers from Queen Mary University of London found that AI-generated voices are now virtually indistinguishable from real human voices.