Check out the on-demand sessions from the Low-Code/No-Code Summit to learn how to successfully innovate and achieve efficiency by upskilling and scaling citizen developers. Watch now.
On Monday, Intel introduced FakeCatcher, which it says is the first real-time detector of deepfakes — that is, synthetic media in which a person in an existing image or video is replaced with someone else’s likeness.
Intel claims the product has a 96% accuracy rate and works by analyzing the subtle “blood flow” in video pixels to return results in milliseconds.
Ilke Demir, senior staff research scientist in Intel Labs, designed FakeCatcher in collaboration with Umur Ciftci from the State University of New York at Binghamton. The product uses Intel hardware and software, runs on a server and interfaces through a web-based platform.
Intel’s deepfake detector is based on PPG signals
Unlike most deep learning-based deepfake detectors, which look at raw data to pinpoint inauthenticity, FakeCatcher is focused on clues within actual videos. It is based on photoplethysmography, or PPG, a method for measuring the amount of light that is absorbed or reflected by blood vessels in living tissue. When the heart pumps blood, it goes to the veins, which change color.
Intelligent Security Summit
Learn the critical role of AI & ML in cybersecurity and industry specific case studies on December 8. Register for your free pass today.
“You cannot see it with your eyes, but it is computationally visible,” Demir told VentureBeat. “PPG signals have been known, but they have not been applied to the deepfake problem before.”
With FakeCatcher, PPG signals are collected from 32 locations on the face, she explained, and then PPG maps are created from the temporal and spectral components.
“We take those maps and train a convolutional neural network on top of the PPG maps to classify them as fake and real,” Demir said. “Then, thanks to Intel technologies like [the] Deep Learning Boost framework for inference and Advanced Vector Extensions 512, we can run it in real time and up to 72 concurrent detection streams.”
Detection increasingly important in face of growing threats
Deepfake detection has become increasingly important as deepfake threats loom, according to a recent research paper from Eric Horvitz, Microsoft’s chief science officer. These include interactive deepfakes, which offer the illusion of talking to a real person, and compositional deepfakes, where bad actors create many deepfakes to compile a “synthetic history.”
And back in 2020, Forrester Research predicted that costs associated with deepfake scams would exceed $250 million.
Most recently, news about celebrity deepfakes has proliferated. There’s the Wall Street Journal coverage of Tom Cruise, Elon Musk and Leonardo DiCaprio deepfakes appearing unauthorized in ads, as well as rumors about Bruce Willis signing away the rights to his deepfake likeness (not true).
On the flip side, there are many responsible and authorized use cases for deepfakes. Companies such as Hour One and Synthesia are offering deepfakes for enterprise business settings — for employee training, education and ecommerce, for example. Or, deepfakes may be created by users such as celebrities and company leaders who want to take advantage of synthetic media to “outsource” to a virtual twin. In those cases, there is hope that a way to provide full transparency and provenance of synthetic media will emerge.
Demir said that Intel is conducting research but it is only in its beginning stages. “FakeCatcher is a part of a bigger research team at Intel called Trusted Media, which is working on manipulated content detection — deepfakes — responsible generation and media provenance,” she said. “In the shorter term, detection is actually the solution to deepfakes — and we are developing many different detectors based on different authenticity clues, like gaze detection.”
The next step after that will be source detection, or finding the GAN model that is behind each deepfake, she said: “The golden point of what we envision is having an ensemble of all of these AI models, so we can provide an algorithmic consensus about what is fake and what is real.”
History of challenges with deepfake detection
Unfortunately, detecting deepfakes has been challenging on several fronts. According to 2021 research from the University of Southern California, some of the datasets used to train deepfake detection systems might underrepresent people of a certain gender or with specific skin colors. This bias can be amplified in deepfake detectors, the coauthors said, with some detectors showing up to a 10.7% difference in error rate depending on the racial group.
And in 2020, researchers from Google and the University of California at Berkeley showed that even the best AI systems trained to distinguish between real and synthetic content were susceptible to adversarial attacks that lead them to classify fake images as real.
In addition, there is the continuing cat-and-mouse game between deepfake creators and detectors. But Demir said that at the moment, Intel’s FakeCatcher cannot be outwitted.
“Because the PPG extraction that we are using is not differentiable, you cannot just plug it into the loss function of an adversarial network, because it doesn’t work and you cannot backpropagate if it’s not differentiable,” she said. “If you don’t want to learn the exact PPG extraction, but want to approximate it, you need huge PPG datasets, which don’t exist right now — there are [datasets of] 30-40 people that are not generalizable to the whole.”
But Rowan Curran, AI/ML analyst at Forrester Research, told VentureBeat by email that “we are in for a long evolutionary arms race” around the ability to determine whether a piece of text, audio or video is human-generated or not.
“While we’re still in the very early stages of this, Intel’s deepfake detector could be a significant step forward if it is as accurate as claimed, and specifically if that accuracy does not depend on the human in the video having any specific characteristics (e.g. skin tone, lighting conditions, amount of skin that can be see in the video),” he said.