AI agents, whether developed in Microsoft Copilot Studio or on alternative platforms, are becoming a powerful means for organizations to create custom solutions designed to enhance productivity and automate organizational processes by seamlessly integrating with internal data and systems.

From a security research perspective, this shift introduces a fundamental change in the threat landscape. As Microsoft Defender researchers evaluate how agents behave under adversarial pressure, one risk stands out: once deployed, agents can access sensitive data and execute privileged actions based on natural language input alone. If an threat actor can influence how an agent plans or sequences those actions, the result may be unintended behavior that operates entirely within the agent’s allowed permissions, which makes it difficult to detect using traditional controls.

To address this, it is important to have a mechanism for verifying and controlling agent behavior during runtime, not just at build time.

By inspecting agent behavior as it executes, defenders can evaluate whether individual actions align with intended use and policy. In Microsoft Copilot Studio, this is supported through real-time protection during tool invocation, where Microsoft Defender performs security checks that determine whether each action should be allowed or blocked before execution. This approach provides security teams with runtime oversight into agent behavior while preserving the flexibility that makes agents valuable.

In this article, we examine three scenarios inspired by observed and emerging AI attack techniques, where threat actors attempt to manipulate agent tool invocation to produce unsafe outcomes, often without the agent creator’s awareness. For each scenario, we show how webhook-based runtime checks, implemented through Defender integration with Copilot Studio, can detect and stop these risky actions in real time, giving security teams the observability and control needed to deploy agents with confidence.

Topics, tools, and knowledge sources: How AI agents execute actions and why attackers target them

Figure 1: A visual representation of the 3 elements Copilot Studio agents relies on to respond to user prompts.

Microsoft Copilot Studio agents are composed of multiple components that work together to interpret input, plan actions, and execute tasks. From a security perspective, these same components (topics, tools, and knowledge sources) also define the agent’s effective attack surface. Understanding how they interact is essential to recognizing how attackers may attempt to influence agent behavior, particularly in environments that rely on generative orchestration to chain actions at runtime. Because these components determine how the agent responds to user prompts and autonomous triggers, crafted input becomes a primary vector for steering the agent toward unintended or unsafe execution paths.

When using generative orchestration, each user input or trigger can cause the orchestrator to dynamically build and execute a multi-step plan, leveraging all three components to deliver accurate and context-aware results.

1. Topics are modular conversation flows triggered by specific user phrases. Each topic is made up of nodes that guide the conversation step-by-step, and can include actions, questions, or conditions.

2. Tools are the capabilities the copilot can call during a conversation, such as connector actions, AI builder models, or generative answers. These can be embedded within topics or executed independently, giving the agent flexibility in how it handles requests.

3. Knowledge sources enhance generative answers by grounding them in reliable enterprise content. When configured, they allow the copilot to access information from Power Platform, Dynamics 365, websites, and other external systems, ensuring responses are accurate and contextually relevant. Read more about Microsoft Copilot Studio agents here.

Understanding and mitigating potential risks with real-time protection in Microsoft Defender

In the model above, the agent’s capabilities are effectively equivalent to code execution in the environment. When a tool is invoked, it can perform real-world actions, read or write data, send emails, update records, or trigger workflows – just like executing a command inside a sandbox where the sandbox is a set of all the agent’s capabilities. This means that if an attacker can influence the agent’s plan, they can indirectly cause the execution of unintended operations within the sandbox. From a security lens:

• The risk is that the agent’s orchestrator depends on natural language input to determine which tools to use and how to use them. This creates exposure to prompt injection and reprogramming failures, where malicious prompts, embedded instructions, or crafted documents can manipulate the decision-making process.

• The exploit occurs when these manipulated instructions lead the agent to perform unauthorized tool use, such as exfiltrating data, carrying out unintended actions, or accessing sensitive resources, without directly compromising the underlying systems.

Because of this, Microsoft Defender treats every tool invocation as a high-value, high-risk event,** **and monitors it in real time. Before any tool, topic, or knowledge action is executed, the Copilot Studio generative orchestrator initiates a webhook call to Defender. This call transmits all relevant context for the planned invocation including the current component’s parameters, outputs from previous steps in the orchestration chain, user context, and other metadata.

Defender analyzes this information, evaluating both the intent and destination of every action, and decides in real time whether to allow or block the action, providing precise runtime control without requiring any changes to the agent’s internal orchestration logic.

By viewing tools as privileged execution points and inspecting them with the same rigor we apply to traditional code execution, we can give organizations the confidence to deploy agents at scale – without opening the door to exploitation.

Below are three realistic scenarios where our webhook-based security checks step in to protect against unsafe actions.

Malicious instruction injection in an event-triggered workflow

Consider the following business scenario: a finance agent is tasked with generating invoice records and responding to finance-related inquiries regarding the company. The agent is configured to automatically process all messages sent to invoice@contoso.com** **mailbox using an event trigger. The agent uses the generative orchestrator, which enables it to dynamically combine tools, topics, and knowledge in a single execution plan.

In this setup:

• Trigger: An incoming email to invoice@contoso.com starts the workflow.

• Tool: The CRM connector is used to create or update a record with extracted payment details.

• Tool: The email sending tool sends confirmation back to the sender.

• Knowledge: A company-provided finance policy file was uploaded to the agent so it can answer questions about payment terms, refund procedures, and invoice handling rules.

The instructions that were given to the agent are for the agent to only handle invoice data and basic finance-related FAQs, but because generative orchestration can freely chain together tools, topics, and knowledge, its plan can adapt or bypassed based on the content of the incoming email in certain conditions.

A malicious external sender could craft an email that appears to contain invoice data but also includes hidden instructions telling the agent to search for unrelated sensitive information from its knowledge base and send it to the attacker’s mailbox. Without safeguards, the orchestrator could interpret this as a valid request and insert a knowledge** **search step into its multi-component plan, followed by an email sent to the attacker’s address with the results.

Before the knowledge component is invoked, MCS sends a webhook request to our security product containing:

• The target action (knowledge search).

• Search query parameters derived from the orchestrator’s plan.

• Outputs from previous orchestration steps.

• Context from the triggering email.

Agent Runtime Protection analyzes the request and blocks the invocation before it executes, ensuring that the agent’s knowledgebase is never queried with the attacker’s input.

This action is logged in the Activity History, where administrators can see that the invocation was blocked, along with an error message indicating that the threat-detection controls intervened:

In addition, an XDR informational alert will be triggered in the security portal to keep the security team aware of potential attacks (even though this specific attack was blocked):

Consider that an organizational agent is connected to the company’s cloud-based SharePoint environment, which stores internal documents. The agent’s purpose is to retrieve documents, summarize their content, extract action items, and send these to relevant recipients.

To perform these tasks, the agent uses:

• Tool A – to access SharePoint files within a site (using the signed-in user’s identity)

A malicious insider edits a SharePoint document that they have permission to, inserting crafted instructions intended to manipulate the organizational agent’s behavior.

When the crafted file is processed, the agent is tricked into locating and reading the contents of a sensitive file, transactions.pdf, stored on a different SharePoint file the attacker cannot directly access but that the connector (and thus the agent) is permitted to access. The agent then attempts to send the file’s contents via email to an attacker-controlled domain.

At the point of invoking the email-sending tool, Microsoft Threat Intelligence detects that the activity may be malicious and blocks the email, preventing data exfiltration.

Capability reconnaissance attempt on agent

A publicly accessible support chatbot is embedded on the company’s website without requiring user authentication. The chatbot is configured with a knowledge base that includes customer information and points of contact.

An attacker interacts with the chatbot using a series of carefully crafted and sophisticated prompts to probe and enumerate its internal capabilities. This reconnaissance aims to discover available tools and potential actions the agent can perform, with the goal of exploiting them in later interactions.

After the attacker identifies the knowledge sources accessible to the agent, they can extract all information from those sources, including potentially sensitive customer data and internal contact details, causing it to perform unintended actions.

Microsoft Defender detects these probing attempts and acts to block any subsequent tool invocations that were triggered as a direct result, preventing the attacker from leveraging the discovered capabilities to access or exfiltrate sensitive data.

Final words

Securing Microsoft Copilot Studio agents during runtime is critical to maintaining trust, protecting sensitive data, and ensuring compliance in real-world deployments. As demonstrated through the above scenarios, even the most sophisticated generative orchestrations can be exploited if tool invocations are not carefully monitored and controlled.

Defender’s webhook-based runtime inspection combined with advanced threat intelligence, organizations gain a powerful safeguard that can detect and block malicious or unintended actions as they happen, without disrupting legitimate workflows or requiring intrusive changes to agent logic (see more details at the ‘Learn more’ section below).

This approach provides a flexible and scalable security layer that evolves alongside emerging attack techniques and enables confident adoption of AI-powered agents across diverse enterprise use cases.

As you build and deploy your own Microsoft Copilot Studio agents, incorporating real-time webhook security checks will be an essential step in delivering safe, reliable, and responsible AI experiences.

*This research is provided by Microsoft Defender Security Research with contributions from Dor Edry, Uri Oren. *

Learn more

• Review our documentation to learn more about our real-time protection capabilities and see how to enable them within your organization.

• Learn more about securing Copilot Studio agents with Microsoft Defender

• Learn more about Protect your agents in real-time during runtime (Preview) – Microsoft Defender for Cloud Apps | Microsoft Learn

• Explore how to build and customize agents with Copilot Studio Agent Builder