Scammers Are Using AI – Can Security Keep Up?

AI is making it easier to write, create images, translate languages and automate work.

The same capabilities can also help scammers.

Artificial intelligence did not invent phishing, fake invoices, romance scams or phone calls from someone pretending to be a family member. Those methods are old.

What has changed is how quickly they can become personalized, convincing and scalable.

The FBI’s 2025 Internet Crime Report included a dedicated section on AI for the first time. It recorded 22,364 complaints containing AI-related information, with adjusted reported losses of more than $893 million. The FBI also makes an important distinction: AI is a tool used in these crimes, not necessarily the sole cause of every reported loss.

That changes an old security rule.

Poor grammar, strange wording or an unnatural voice used to be warning signs.

Today, the scam can sound exactly as it should.

The scammer does not need to hack your computer

One of the most important changes is that AI does not need to break through an advanced technical security system to cause damage.

Sometimes, it only needs to help the scammer convince you.

A well-written email can imitate a manager. A message can contain details about your family. A fake customer-service representative can sound polished and professional.

Voice cloning makes the problem even harder.

The FBI has described how AI can be used to create convincing social profiles, personalized conversations, executive-style emails and synthetic voices. The FTC has also warned that scammers can clone a family member’s voice from a short audio sample taken from the internet.

That means “I heard his voice” is no longer the proof it once was.

AI automates credibility

AI can also reduce the amount of work required to build the story behind a scam.

Public information from social media, company websites and other open sources can be used to create a more believable context.

Who handles the company’s finances? Who is on vacation? Which suppliers does the company use? How does the CEO normally communicate? Which family members appear publicly online?

The scammer does not need to know everything.

A handful of accurate details may be enough to make everything else feel authentic.

And once the method works, it can be scaled to far more targets than if every message had to be written manually.

That may be the biggest change:

AI does not just automate text. It can automate credibility.

OpenAI has documented organized scam operations using AI for fake identities, translation, personalized messaging and administration. The company has also described how threat actors often combine multiple AI models with conventional websites and social platforms instead of relying on a single service.

That becomes important when we ask who is ultimately responsible for stopping the abuse.

AI companies are not standing still

It would be misleading to suggest that the largest AI companies are simply racing to build more powerful models while leaving security to someone else.

There are clear examples of the opposite.

On August 18, OpenAI said it had temporarily slowed parts of its model-development work after indications that upcoming models could reach a critical level of cyber capability under the company’s Preparedness Framework.

Reinforcement-learning training for models intended for release was paused for two weeks. OpenAI also said its largest planned frontier RL run remained on hold while smaller tests were conducted and security in its research environments was strengthened. The company says those measures have cost both time and resources.

That is a concrete example of security being allowed to take priority over speed.

OpenAI has also shut down accounts linked to a Cambodia-based scam operation involving investment scams, romance scams, gambling-related fraud and identity impersonation.

Google is trying to make synthetic content visible

Google is approaching part of the problem from another direction.

SynthID embeds signals into AI-generated content so that images, video and audio can be more easily identified as synthetic. Google said this year that the technology had been applied to more than 100 billion images and videos, as well as the equivalent of roughly 60,000 years of generated audio.

The company is also combining this approach with C2PA Content Credentials, which can provide information about how digital content was created or modified.

But Google has acknowledged a limitation that goes directly to the wider issue:

these systems become far more useful when more organizations participate.

That matters.

A standard has limited value if it only works inside one company’s ecosystem.

Anthropic shows why safety tests also need safeguards

Anthropic uses safety classifiers, among other tools, to detect and block dangerous cyber activity.

But the company has also disclosed incidents in which the safety testing itself went wrong.

After reviewing more than 141,000 cybersecurity evaluations, Anthropic identified three incidents in which Claude models unintentionally gained internet access from test environments and subsequently accessed real systems belonging to three organizations.

The details matter.

Anthropic says the models had been instructed to perform capture-the-flag exercises and mistakenly believed the real systems were part of the simulation. There was no evidence that the models were deliberately trying to “escape” their test environment or pursue an independent goal.

Normal production safeguards were also not enabled during those evaluations.

Even so, the incidents demonstrate something important:

Testing highly capable AI agents can itself become a security risk if the environment around the model is not secured to the same standard as the model.

Anthropic halted its cyber evaluations when the problems were discovered and later described changes to its monitoring and testing environments.

The fact that the company publicly disclosed the incidents is itself an important part of responsible safety work.

SpaceXAI is documenting its safety work too

SpaceXAI/xAI has published a Frontier AI Framework covering risks including malicious use and loss of control, and describing how risks should be evaluated throughout model development and deployment.

The company’s safety documentation also says evaluations are used from pre-training through deployment.

When Grok 4.6 was released on August 12, SpaceXAI said the model had undergone its broadest set of pre-deployment capability and safeguard tests to date, together with extensive post-deployment and third-party testing.

That is the company’s own description and should be treated as such.

This matters for balance.

Safety work is also taking place at companies simultaneously pursuing extremely aggressive AI-development and infrastructure strategies.

The more important question is how well safety scales when capability does.

Detecting the problem is not the same as stopping it

A recent review by the nonprofit Guidelight AI Standards adds another dimension.

Guidelight, founded by two former OpenAI figures with backgrounds in safety and policy, assessed publicly documented control procedures at OpenAI, Anthropic, Google, xAI and Meta.

OpenAI and Anthropic received the highest overall grades at C+, while Google received D+, xAI D− and Meta F. The assessment covered areas including monitoring, external review and the ability to restrict a system if it begins operating outside intended boundaries.

But those grades require an important caveat:

Guidelight evaluates what companies have publicly documented.

A low score therefore does not automatically mean that a company lacks internal safeguards.

TechCrunch reported on August 22 that few of the reviewed AI labs had publicly documented detailed containment plans. Guidelight defines such plans as predetermined procedures covering which permissions should be removed, how a model may continue operating and when it should be taken completely offline if it attempts to circumvent controls.

OpenAI told TechCrunch that its internal processes can restrict permissions, pause workloads, limit deployment or take a model fully offline, and that such processes have been used.

Google, meanwhile, argued that Guidelight’s review does not capture the full scope of its internal safety work.

That makes the conclusion more nuanced:

The absence of public documentation is not the same as the absence of security.

But it does make it harder for researchers, regulators, customers and the public to assess how prepared the industry really is once something has already gone wrong.

The industry appears to have made significant progress in detecting problems.

The next question is how clearly it can demonstrate that those problems can also be stopped.

Is it enough for every company to build its own safeguards?

This is where the broader question begins.

Common frameworks already exist.

NIST’s AI Risk Management Framework and its generative-AI profile offer structures for identifying and managing AI risks, but their use is voluntary.

The European Union has gone further.

The AI Act imposes binding obligations on providers of general-purpose AI models with systemic risk. These include model evaluations, risk mitigation, serious-incident reporting and cybersecurity requirements covering both the models and the physical infrastructure surrounding them.

OpenAI, Google, Anthropic and Microsoft are among the companies that signed the EU’s GPAI Code of Practice. xAI signed its Safety and Security chapter.

The Code itself is a voluntary mechanism intended to help providers demonstrate compliance with the AI Act. The legal obligations come from the law.

So it would be wrong to say that common rules do not exist.

But there is still no single global security standard that every major AI developer is required to follow in the same way.

And attackers do not have to respect corporate or national boundaries.

OpenAI has also sharpened its public warning.

On August 23, the company’s Chief Global Affairs Officer, Chris Lehane, described AI-driven cybercrime as entering a new phase in which more capable systems could enable significantly more persistent attacks.

OpenAI has already argued for common national security frameworks and greater international coordination around AI standards.

That only strengthens the question of whether company-by-company safeguards will be enough as capability continues to grow.

An attempt to learn from each other’s mistakes

On August 4, another potentially important initiative emerged.

Within the Open Secure AI Alliance, a proposal was introduced for SAFE — the Shared AI Findings Exchange.

The idea is that organizations could confidentially share security incidents and near misses, identify recurring control failures and turn those lessons into common tests, rules and defensive methods.

The Linux Foundation has emphasized that SAFE is still a proposal for open discussion, not a finished standard.

NVIDIA, Cisco, CrowdStrike, Hugging Face and Red Hat are among the organizations involved in the initial work.

NVIDIA describes the initiative as a way to turn incidents involving agentic AI into shared protection for the wider ecosystem.

The interesting part is not really the name SAFE.

It is the principle behind it.

Aviation learns from accidents and near misses.

Traditional cybersecurity has long relied on systems for sharing vulnerabilities and threat intelligence.

As AI systems become more autonomous, the same principle may become far more important here as well.

What can you do yourself?

The industry can build new standards and technical safeguards, but individuals still need ways to deal with the scam that arrives today.

• Do not treat a voice, image or video as sufficient proof of identity.
• Verify unexpected payments or sensitive instructions through a separate communication channel.
• Call back using a phone number you already know instead of one supplied in the message.
• Be especially cautious when someone creates urgency, demands secrecy or suddenly changes payment details.
• Use multifactor authentication or passkeys wherever possible.
• Businesses should require additional verification for large payments and changes to account information.
• Families can agree on a simple verification question or another method for confirming unexpected emergency calls.

The FTC’s advice on suspected voice cloning follows the same principle: do not trust the voice alone. Contact the person through a communication method you already know.

Can security keep up?

AI did not invent fraud.

It has made it easier to make fraud more personalized, convincing and scalable.

At the same time, there are clear signs that the AI industry is taking security seriously.

OpenAI has slowed development when cyber capabilities required stronger safeguards.

Google is building systems for provenance and synthetic-content identification.

Anthropic is publishing incidents and changing its testing environments.

SpaceXAI is documenting its safety processes and evaluations.

NVIDIA is working with other organizations on shared incident learning.

The European Union has introduced binding requirements for the most advanced general-purpose models.

The industry deserves credit for that work.

But the next stage may require more than every company building strong defenses around its own systems.

A scammer can move between models, services and platforms.

An AI agent can interact with systems developed by someone else.

And an incident at one company may contain lessons the rest of the industry needs that same day — not six months later.

The problem may therefore not be that security is missing.

The problem is that security is still fragmented while AI capability is advancing at extraordinary speed.

Which leaves the most important question:

When AI capability is growing at record speed, how do we make sure protection develops just as quickly?

--------------------------------------------------

💬 What do you think?

AI companies are building increasingly sophisticated safeguards, but attackers can move between models, services and platforms.

Is it enough for each company to build its own protection — or does the AI industry need a shared baseline for security?

Share your thoughts in the comments.

📚 Related Articles