In my long career of safeguarding digital assets, I’ve seen technology shifts come and go, but nothing compares to the disruptive force of AI and large language models (LLMs). AI is quite literally a double-edged sword when it comes to cybersecurity. The same capabilities that allow us to automate threat detection and write secure code faster are being weaponized by adversaries to create a new generation of cyberthreats. For any CISO or security leader, our primary focus must shift; we are no longer just fighting human adversaries, we are fighting AI-augmented, automated threat agents.

The offensive edge

We are witnessing a drastic reduction in the cost and expertise required for cybercrime. What used to take a skilled human weeks can now be orchestrated by an AI agent in hours, driving what we call cyberthreat inflation.

Self-modifying, evasive malware

More recently, the researchers at Google Threat Intelligence Group (GTIG) identified a disturbing new trend: malware that uses LLMs during execution to dynamically alter its own behavior and evade detection. This is not pre-generated code, this is code that adapts mid-execution.

In June 2025, GTIG identified an experimental malware called PROMPTFLUX, which connects to a commercial LLM API (such as the Gemini API) to request new code-obfuscation and evasion scripts on the fly. This technique, which allows for just-in-time code generation, represents a significant step toward autonomous and adaptive malware.

In another case, GTIG identified PROMPTSTEAL, which was being used by the Russia-linked APT28 group against Ukraine. This malware queries an LLM (the Qwen2.5-Coder model on Hugging Face) to generate reconnaissance commands instead of having them hard-coded.

Some analysis projects that the percentage of all malware detections with an LLM contribution has soared from just 2% in 2021 to a projected 50% by 2025.

Deepfake-driven financial fraud

On the other hand, generative AI has amplified social engineering through hyper-realistic deepfakes. Deepfake-based scams have skyrocketed, enabling voice and video forgeries that deceive even seasoned professionals. A dramatic example occurred when a Ferrari executive was nearly fooled by a voice-cloned deepfake of the CEO.

Even more recently, an employee at a global design firm was fooled by an AI-generated video call impersonating the company’s CFO and other executives, resulting in a $25 million losstransferred to the fraudsters.

The defensive edge

If adversaries are operating at AI speed, our defenses must too. The silver lining of this dual-use dynamic is that the most powerful LLMs are also being harnessed by defenders to create fundamentally new security capabilities.

Vulnerability detection: From zero-days to autonomous pentesting

LLMs’ semantic code understanding and contextual reasoning offer a significant advantage over traditional, signature-based static analyzers, especially in the discovery of unknown threats before malicious actors find and exploit them.

LLMs have shown extraordinary potential in identifying unknown, unpatched flaws (zero-days). These models significantly outperform conventional static analyzers, particularly in uncovering subtle logic flaws and buffer overflows in novel software. For instance, Google’s Big Sleep project used an LLM to identify a zero-day vulnerability in the critical SQLite database used across the industry.

Another example is XBOW, which is an autonomous AI penetration testing agent that leverages LLMs to simulate real-world attacks the same way a human counterpart would do. XBOW achieved the #1 spot on the HackerOne US Leaderboard, demonstrating that AI can match and, in some benchmarks, surpass expert human hackers in finding a broad range of vulnerabilities (e.g., injection flaws, XSS).

By deploying AI agents like XBOW against our own systems, we can systematically test every endpoint and attack vector. This scales ethical offensive security testing from a periodic audit into an on-demand, pre-production workflow.

On the other hand, Google’s DeepMind has pioneered an AI-powered agent called CodeMender that automatically detects, patches and rewrites vulnerable code to eliminate whole classes of security flaws. This reactive and proactive approach represents a major step forward in defensive automation.

Threat hunting and behavioral analysis

LLMs are transforming threat hunting from a manual, keyword-based search to an intelligent, contextual query process that focuses on behavioral anomalies. By processing and correlating massive, unstructured datasets such as network logs, security incident reports and threat intelligence feeds (OSINT), LLMs gain the contextual understanding necessary to generate high-confidence hypotheses about potential threats.

For example, Google Threat Intelligence with Gemini enhances this process by providing essential contextual grounding with data from multiple sources like VirusTotal and Mandiant to determine, in real-time, which global threat actor tactics, techniques and procedures (TTPs) that are most relevant to a particular industry or organization and prioritize what’s most important from tens of thousands of reports.

Furthermore, Palantir’s AIP (Artificial Intelligence Platform) utilizes LLMs to analyze anomalous user or system behavior across vast data lakes, identifies subtle indicators of compromise that span different systems and suggests a prioritized attack path for the human analyst to investigate.

Governance, compliance and regulation

LLMs also offer a profound scalable solution to managing non-technical risk compliance, privacy and regulatory complexity, which represent significant financial and legal liability. LLMs can be leveraged to shift governance from periodic audits to continuous policy enforcement.

For example, Microsoft’s Compliance Manager can ingest and understand vast amounts of unstructured data, such as internal security policies, legal contracts and external regulatory texts like GDPR or the EU AI Act. It then automatically maps the requirements of one framework to another, dramatically reducing the manual effort needed to prove compliance during an audit.

IBM’s Watsonx applies LLM capabilities for compliance and risk management. For a financial services firm, this might mean the model continuously scans new trading application code to verify that all data handling aligns with HIPAA or PCI DSS before deployment, automatically flagging any deviation from established policy.

Bringing it all together

As AI systems continue to advance from generative models to autonomous agents, their dual-use nature can’t be ignored. The same tools that help defenders accelerate incident response can also empower attackers to craft deepfakes, launch social engineering campaigns and more. This tension isn’t a temporary byproduct of innovation, but it is a structural reality of AI’s rapid evolution.

Ultimately, the challenge isn’t to halt AI progress but to guide it responsibly. That means building guardrails into models, improving transparency and developing governance frameworks that keep pace with emerging capabilities. It also requires organizations to rethink security strategies, recognizing that AI is both an opportunity and a risk multiplier.

In the end, AI’s impact on cybersecurity will be shaped not just by what these systems can do, but by how we choose to use them. The future will belong to those who can harness AI’s defensive power while staying vigilant against its offensive potential.

**This article is published as part of the Foundry Expert Contributor Network. **Want to join?

SUBSCRIBE TO OUR NEWSLETTER

From our editors straight to your inbox

Get started by entering your email address below.