How deepfakes are actually getting past your defenses. „Deepfake detection is a losing game.”
The main threat to banks right now isn’t the quality of the deepfake. It’s that fraudsters are bypassing live verification altogether and tampering with the app to inject whatever image or video they want.
an article written by André Ferraz, CEO & Co-founder at Incognia
Last month, my colleague Eduardo posted a deepfake video on LinkedIn, created with tools that anyone can buy for just a few dollars. It went viral on LinkedIn and X, and has over 5 million views.
The comments were full of skeptics: „This won’t bypass security” and „What about live camera capture?” and „Three fingers in front of the face will stop this.”
We took those challenges and proved them wrong. But that’s not the main point I want to make here.
Whether deepfakes look convincing isn’t really the question anymore. They do. Eduardo’s video was made with cheap tools and not a lot of time. If you look close enough, you can probably tell it’s not real. But these tools are only getting better, and fraudsters will always invest in the best ones.
They’re getting so good that even Hany Farid, a Berkeley professor who specializes in deepfakes, says he can no longer tell the difference just by looking at them.
That all matters. But the main threat to banks right now isn’t the quality of the deepfake. It’s that fraudsters are bypassing live verification altogether and tampering with the app to inject whatever image or video they want.
This is already happening at banks globally
Judging by the LinkedIn comments, a lot of people still think deepfakes aren’t good enough to pass verification at a bank. But they are. And they already have.
A few examples:
In Hong Kong, a gang took stolen identity cards and used deepfakes to put their own faces on them. They applied for 44 bank accounts. 30 got through KYC. From there they applied for loans and credit cards, stealing over $860,000. That case was part of a broader crackdown: 503 arrests, close to $200 million stolen in total.
In Israel, one person took leaked ID photos, turned them into deepfake videos, and used AI agents to open bank accounts and credit cards under other people’s names. Hundreds of accounts. Around 120 victims so far. All from one person.
In Brazil, police broke up a group that was using deepfakes to break into bank accounts across multiple institutions. Over 550 attempts. They used fake images of real account holders to get past verification and add new devices to existing accounts. Over four years, the group moved about $20 million through the accounts. Banks caught some of it, but not all.
How they get past defenses
There are two ways this typically happens.
A presentation attack, where someone holds up a fake image or video in front of the camera.
And image injection, where the fraudster bypasses the camera entirely and injects the image directly into the app. This is what we’re seeing a lot more of.
A lot of app cloners now include features that let you replace the live camera feed with any image you want. A pre-recorded photo or video, an AI-generated selfie, whatever. The platform thinks it’s getting a live capture. It’s actually getting whatever the fraudster feeds it.
We’ve seen this firsthand. A platform required live camera selfies for authentication. But then they noticed the exact same image showing up across different accounts, at different times. If the camera is actually live, that’s impossible. Fraudsters were bypassing it entirely.
The tools to do this aren’t hard to find. App cloners with image injection capabilities are available on public app stores. No coding required. No technical skill needed.
We’ve tested some of the largest banks in the world for this vulnerability. The vast majority were vulnerable.
It goes beyond faces
You hear „deepfake” and probably think of someone faking a face to bypass biometrics. That’s one use case. But AI-generated image fraud is much broader than that.
For banks and fintechs, that includes KYC document verification, proof of address, identity document uploads. AI-generated passports have already passed KYC checks.
But this isn’t just a financial services problem. We’re seeing it show up across other industries too.
In food delivery, fraudsters are using AI to alter images of their orders to claim refunds. We saw a case where a single device was accessing 237 different accounts, racking up nearly $5,000 in transactions with over $4,000 refunded. 82% of purchases, essentially free.
On rental platforms, AI-generated images of property damage are being used to file false claims. UK insurers reported a record £230 million in claims attributed to AI-generated images.
I experienced this one myself.
I rented a property for a weekend, and after I left, the host claimed I had destroyed a tabletop and demanded $6,000 using fake images. We disputed it, and the platform had to send someone to the property for an in-person audit to verify the damage wasn’t real. The host was banned. But the cost of physically verifying something that was faked digitally was significant.
The point is, if any part of your verification process relies on images, it’s vulnerable to fraud.
Deepfake detection is a losing game
Right now, there is no reliable and sustainable way to detect whether an image or video was generated by AI. Whatever detection method you build today, the models will outpace it quickly.
The investment going into improving generation models is massive. Some of the largest companies in the world are competing to make these models better. The teams trying to build detection are working with a fraction of those resources. That’s an asymmetric fight, and from what I’ve seen, the detection side isn’t winning.
After Eduardo’s video went viral, a lot of people reposted it saying they had a solution that could detect deepfakes. His response was simple: it’s a cat and mouse game, and if your strategy is to identify whether an image is AI-generated, you’re not going to be able to keep up.
We have to move past trying to spot inconsistencies in the media itself. The approach has to analyze from multiple angles and multiple variables, not just look at the image and try to decide if it’s real.
How to protect against deepfakes and AI-generated media
For image injection specifically, the answer is live camera collection combined with strong app tamper detection.
If you can detect that the app has been cloned or modified, you catch the injection before the fake image ever reaches your system. The deepfake itself doesn’t matter if it never gets in.
For the broader AI-generated fraud problem, you have to move beyond analyzing the media itself. You need multiple layers of defense, and they have to be grounded in what’s actually happening in the physical world. That’s the only way to stay ahead of this.
There’s only one thing AI won’t be able to fake at scale: the physical world. If you can collect data about what’s actually happening in the real world, you’re in a much better position than anyone trying to out-detect the models.
Eduardo and I did a full session on this topic where we go deeper into how these attacks work and how to defend against them. You can watch the recording here:
