A Fraudster’s Paradise – O’Reilly

Darkish net discussion board posts talked about the phrase “AI agent” way more within the second half of 2025 than within the first half. Might this imply that fraudsters are charmed by the AI hype? Or is AI really a sport changer for cybercrime? AI-related discussions—evident each in what “the unhealthy guys” are saying and in what fraud-fighters are exploring—are in all places. In actual fact, a Visa PERC evaluation evaluating knowledge from June to November 2025 to the six months earlier than that discovered an improve of greater than 450% in darkish net agentic AI-related posts.

Documented monetary losses from deepfake-enabled fraud exceeded $200 million in Q1 2025 alone. And Visa reported a 25% improve within the second half of the yr in malicious bot-initiated transactions. There’s AI-generated rip-off and fraud content material in all places you look, and it doesn’t at all times give Eric Clapton six fingers to make it apparent that one thing’s off:

None of that is occurring in a vacuum. On-line fraudsters have embraced GenAI with open arms in a approach that has modified the form of the net fraud panorama. On the identical time, bizarre customers are leaning into the form of dishonest that GenAI makes straightforward.

How can the monetary and digital ecosystem struggle again towards this new wave of AI-powered fraud? Studying from previous expertise, collaboration and data sharing between fraud-fighting professionals is vital to our collective means to brace for influence. This quick weblog put up is a name for harnessing the facility of AI for enhancing the know-how of digital defenders, as a option to strengthen our total protection.

Bear in mind COVID?

AI isn’t the primary time that the principles of cybercrime have modified dramatically (and it gained’t be the final). For instance, again in 2020, throughout the first COVID-19 lockdowns, work-from-home schemes exploded, together with first-party fraud, phishing scams, and extra. On the time, the neighborhood responded successfully. Groups of fraud-fighting consultants joined forces to fulfill just about, study the brand new terrain and finally write a examine information that might empower organizations to defend communities from a surge of digital fraud. Our ebook Sensible Fraud Prevention was the results of that collaboration, and it shortly grew to become a useful coaching useful resource to many groups within the ecosystem of their struggle towards on-line monetary crime.

Right this moment, a brand new wave of fraud is rising, powered by AI and particularly GenAI. Whereas some AI-powered initiatives are right here just for a short time, others have gotten really highly effective and harmful instruments in nefarious palms. It’s solely pure that the skilled neighborhood will as soon as once more regroup to type a playbook towards these tendencies.

As we interview consultants in various fields for our subsequent ebook, The Fraud Fighter’s AI Playbook (with coauthor Chen Zamir, now in early launch on O’Reilly), we’re constructing an image of the ways in which GenAI is altering not simply the methods fraudsters function on-line but in addition the form of the net fraud panorama itself. We’re seeing, too, how very important it’s that fraud fighters themselves put money into exploring and utilizing this expertise to spice up their success, technique, and inside popularity in their very own corporations.

Extra fraudsters, doing extra hurt

There are extra on-line fraudsters at present, finishing up extra fraud assaults, than ever earlier than. Not all of this may be blamed on GenAI. Slightly, GenAI entered right into a fraud world that, looking back, was poised to leverage it for crooked growth.

The COVID-19 pandemic drew many new fraudsters on-line, by way of three principal tracks:

• Digital transformation + time to spare. Retailers, banks, and different organizations needed to cram years of digitisation into weeks or months. It was inevitable that this might lead to some vulnerabilities in processes, insurance policies, or methods. Individuals caught at residence with no work have been tempted to make some cash exploiting these weaknesses and realized quick.
• COVID aid packages. Packages put in place to tide people over the difficulties of the pandemic didn’t at all times embody checks to confirm identities or claims. Those that realized the way to make fraudulent submissions, typically even by creating faux or artificial identities or companies, discovered how straightforward and the way profitable that may be.
• Rip-off compounds. Human trafficking rings pivoted from in-person exploitation to forcing folks on-line to hold out phishing assaults and scams of every kind.

None of those tendencies has disappeared, and the rip-off compounds specifically have expanded massively for the reason that finish of the pandemic. This was the world that GenAI was born into.

With GenAI, way more fraud assaults are doable, at a stage of personalization that might have been inconceivable at scale with out the expertise. Widespread makes use of of GenAI within the wild embody:

• Phishing campaigns, personalised by utilizing open supply details about the goal (sourced by way of GenAI), and the suitable language and cultural touchstones for every sufferer (made straightforward by placing conversations by way of GenAI). Extra typically, the influence is seen in scams of every kind, from romance scams to funding scams to kidnapping, catfishing, blackmail scams and extra—equally personalised. 

• Malware and bot creation to hold out assaults, steal data, arrange faux accounts, and extra. Fraudsters now not want a lot technical means to create malicious packages or automation, enabling extra fraudsters to amp up their attain. 
• Deepfakes. Whether or not for clickbait era or for monetary scams, deepfakes enable attackers to move id validation checks. In surprising circumstances, deepfakes have been even used to faux the trail to a job (and the wage and entry to knowledge that comes with it). Broadly, AI simplifies the method of making faux or artificial identities utilizing open supply data, paired with stolen private credentials.
• Pretend web sites, faux apps, faux promoting. Manipulated content material may be utilized to steer customers to buy nonexistent or low-quality merchandise, however it might additionally goal digital promoting. Main manufacturers are dropping hundreds of thousands of {dollars} to AI schemes that generate views, impressions, or clicks for undeserved advert income. A examine of pretend cellular apps, led by Gilit’s workforce at DoubleVerify, discovered that faux iOS grew to become 3 times extra frequent (accompanied by six occasions extra faux Android apps) in 2025 in comparison with earlier years, a pattern amplified by AI.
• Fast evolution in sophistication and evasion of bot schemes. Agentic AI is getting higher day by day. Which means operators of bot networks are smirking at CAPTCHA challenges (“resolve this puzzle to show you’re not a bot”). Fraud assaults of 2023 had some rookie errors, like bot networks that attempted to move as human beings watching TV content material besides that they have been coming in with machine settings of fridges. 2026 attackers will now not fail at fundamental deception as a result of they’ve AI chatbots (FraudGPT, WormGPT, and so on.) to information them on their approach.

The excellent news—if there’s any actual excellent news—is that no less than these aren’t new varieties of assaults. They’re acquainted assaults, carried out extra convincingly, at far larger scale.

GenAI is a facet hustler’s greatest good friend

It’s not simply devoted fraudsters utilizing GenAI to increase their attain. Bizarre folks use it to stage up their dishonest sport too.

Refund fraud has change into straightforward at a really convincing stage due to GenAI. Many retailers ask for photographic proof that an merchandise has arrived damaged or broken, and with GenAI, that’s one thing that may be faked in seconds. Because the picture is created for a objective from scratch, there’s no option to discover an authentic on-line as proof that it’s a cheat.

Some folks have gotten much more inventive, utilizing the identical form of trick as a part of an insurance coverage declare. Others use GenAI to whip up faux receipts, which they’ll declare again from their firm.

It’s necessary to notice that, as with the skilled use circumstances, it’s not that these cheats are new kinds of assaults. What’s new is the convenience, scale, and class with which they are often carried out.

The bottom shifting underneath our toes

Once we wrote Sensible Fraud Prevention, we included a dialogue of issues like phishing, victim-assisted fraud, refund fraud, and so forth. The give attention to the ebook, although, was on the ways in which fraudsters money out their schemes. Observe the cash, and discover the fraudster.

Now, solely just a little greater than three years after ChatGPT burst into all of our lives, that emphasis has shifted. Conventional third-party fraud, the type you get when a fraudster makes use of your bank card on-line, isn’t even within the prime three fraud considerations. TransUnion studies that the most enterprise loss in 2025 got here from rip-off/approved fraud (24%), adopted carefully by artificial id fraud (20%) and account takeover (20%). That’s the GenAI influence.

There’s a big price ticket connected to numbers like that. The identical report famous that “corporations worldwide misplaced 7.7% of their annual income on common as a result of fraud over the previous yr.” Within the US, it was 9.8%.

There’s additionally a worrying influence on belief. When deepfakes are frequent and convincing, who can ever consider their eyes? Prospects don’t know which websites are actual, which messages are genuine, or which advertisements or provides may be trusted. Companies don’t know which claims are official or how greatest to remain forward of the verification challenges they now face. Marketplaces wrestle to guard consumers from dishonest sellers, and sellers from dishonest consumers, and everybody from exploitation by malicious actors.

Ha! Fraud fighters have GenAI, too

The ray of hope in our analysis is that fraud fighters have GenAI too, and groups are already experimenting in a wide range of methods. Some are taking a look at how they’ll use brokers to increase their open supply analysis to make their choices quicker and extra correct. Others are engaged on the way to craft prompts to assist analyze knowledge or work out tendencies that can be utilized to establish and cease fraud. Nonetheless others are leveraging GenAI to investigate documentation, to pick fakes or alterations. And so forth.

It’s additionally encouraging to see how groups are utilizing GenAI to increase their attain internally inside an organization. In some methods, it’s virtually like fraud departments are getting the assistant they’d at all times wished to do the duties they’d at all times meant to get to—like pulling the info and placing it collectively for a biweekly replace to related stakeholders or creating detailed materials with useful illustrations or graphs for shows to different departments.

It’s inevitable that when a completely new expertise comes alongside, the fraudsters may have an higher hand initially. They aren’t hampered by concerns like regulatory considerations or authorized necessities, and so they don’t care about issues like accountability, accountability, or shopper belief. It’s just about of their job description to disregard these issues, actually.

The fraud-fighting business has been sensibly cautious about understanding the way to establish and make use of GenAI, but it surely’s clear that they’re not standing nonetheless. The groups who do the very best with this evolving problem would be the ones who work carefully and persistently with departments throughout their firm to adapt shortly to the enterprise’s wants—and the way to meet them.