Security Guide
jnMetaCode/shellward
OpenClaw 安全部署指南 / Security deployment guide — help users secure their OpenClaw installation
Report/investigate RUNTIME ACTIVITY of AI agents (Agent 365 / Copilot Studio / M365 Copilot / Work IQ) — agents used, tools/connectors, channels, tokens, prompt/reply content, and Prompt Shield…
$ npx skills add SCStelz/security-investigator --skill ai-agent-activity -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install SCStelz/security-investigator ai-agent-activity --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/SCStelz/security-investigator.git skills-src && mkdir -p .claude/skills && cp -r skills-src/.github/skills/ai-agent-activity .claude/skills/ai-agent-activity && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "ai-agent-activity" agent skill from https://github.com/SCStelz/security-investigator/tree/main/.github/skills/ai-agent-activity into .claude/skills/ai-agent-activity/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "ai-agent-activity", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/SCStelz/security-investigator/tree/main/.github/skills/ai-agent-activityType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add SCStelz/security-investigator --skill ai-agent-activity -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install SCStelz/security-investigator ai-agent-activity --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/SCStelz/security-investigator.git skills-src && mkdir -p .agents/skills && cp -r skills-src/.github/skills/ai-agent-activity .agents/skills/ai-agent-activity && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "ai-agent-activity" agent skill from https://github.com/SCStelz/security-investigator/tree/main/.github/skills/ai-agent-activity into .agents/skills/ai-agent-activity/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "ai-agent-activity", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add SCStelz/security-investigator --skill ai-agent-activity -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install SCStelz/security-investigator ai-agent-activity --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/SCStelz/security-investigator.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/.github/skills/ai-agent-activity .cursor/skills/ai-agent-activity && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "ai-agent-activity" agent skill from https://github.com/SCStelz/security-investigator/tree/main/.github/skills/ai-agent-activity into .cursor/skills/ai-agent-activity/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "ai-agent-activity", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/SCStelz/security-investigator.git --path .github/skills/ai-agent-activity--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add SCStelz/security-investigator --skill ai-agent-activity -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install SCStelz/security-investigator ai-agent-activity --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/SCStelz/security-investigator.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/.github/skills/ai-agent-activity .gemini/skills/ai-agent-activity && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "ai-agent-activity" agent skill from https://github.com/SCStelz/security-investigator/tree/main/.github/skills/ai-agent-activity into .gemini/skills/ai-agent-activity/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "ai-agent-activity", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install SCStelz/security-investigator ai-agent-activityInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add SCStelz/security-investigator --skill ai-agent-activity -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/SCStelz/security-investigator.git skills-src && mkdir -p .github/skills && cp -r skills-src/.github/skills/ai-agent-activity .github/skills/ai-agent-activity && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "ai-agent-activity" agent skill from https://github.com/SCStelz/security-investigator/tree/main/.github/skills/ai-agent-activity into .github/skills/ai-agent-activity/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "ai-agent-activity", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add SCStelz/security-investigator --skill ai-agent-activity -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install SCStelz/security-investigator ai-agent-activity --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/SCStelz/security-investigator.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/.github/skills/ai-agent-activity .opencode/skills/ai-agent-activity && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "ai-agent-activity" agent skill from https://github.com/SCStelz/security-investigator/tree/main/.github/skills/ai-agent-activity into .opencode/skills/ai-agent-activity/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "ai-agent-activity", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
ai-agent-activityReport/investigate RUNTIME ACTIVITY of AI agents (Agent 365 / Copilot Studio / M365 Copilot / Work IQ) — agents used, tools/connectors, channels, tokens, prompt/reply content, and Prompt Shield…
AI Agent Activity is an agent skill from SCStelz/security-investigator. Report/investigate RUNTIME ACTIVITY of AI agents (Agent 365 / Copilot Studio / M365 Copilot / Work IQ) — agents used, tools/connectors, channels, tokens, prompt/reply content, and Prompt Shield jailbreak/XPIA verdicts. Triggers: "agent activity", "AI agent usage", "who is using agents", "agent runtime", "agent telemetry", "agent tool usage", "agent session", "jailbreak activity", "prompt injection activity", "Agent 365 activity", "UnifiedAgentObservability", "CloudAppEvents agents", "CopilotActivity"…
Its SKILL.md is about 17k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.
It sits in Security, covering LLM guardrails and Prompt injection and agent security. It works with Microsoft Copilot Studio and Microsoft 365. The repository describes itself as: Automated security investigation tool using Microsoft MCP Servers, GitHub Copilot, Python Modules and custom copilot-instructions. The licence is MIT.
9 steps, taken from the step headings in SKILL.md.
Read from SKILL.md and the folder at commit 51e1385. It shows what the files ask for, not the result of running them.
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
No scripts in the folder and no shell commands in SKILL.md (its code samples are kql, mermaid and markdown).
From the folder's file list and the shell code blocks in SKILL.md.
Hosts in commands or code, which the agent is likely to contact:
security.microsoft.comAlso links to:
learn.microsoft.comFrom URLs in SKILL.md, links to its own repository left out.
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
AI Agent Activity loads about 17k tokens when it runs. Until then it costs about 205 tokens; SKILL.md has 5,367 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.
The full file from SCStelz/security-investigator at commit 51e1385, republished under its MIT licence (© SCStelz). 5,367 words, ~16,863 tokens.
.claude/skills/ai-agent-activity/SKILL.md (or your agent's skills folder).This skill reports the runtime activity of AI agents built on Agent 365 / Copilot Studio / Microsoft 365 Copilot / Work IQ across a tenant — who invoked which agents, what tools and connectors ran, over which channels, with what token/inference usage, and what the content-safety layer (Prompt Shield jailbreak / XPIA) caught.
It answers "what are the agents actually doing?" — the behavioral counterpart to the configuration-focused ai-agent-posture skill.
ai-agent-posture (config) | ai-agent-activity (this skill — runtime) | |
|---|---|---|
| Question | How are agents configured? (access, tools declared, data sources, credentials) | What are agents doing? (prompts, tool calls, users, channels, safety flags) |
| Primary table | AgentsInfo (Advanced Hunting) | UnifiedAgentObservability (Data Lake) or CloudAppEvents (Defender) |
| Time model | Point-in-time config snapshots | Event stream over a lookback window |
| Use together | Posture flags a broadly-accessible, email-capable agent | Activity shows whether that agent is actually used, by whom, and whether it was jailbroken |
Use them together: run ai-agent-posture to find the risky configurations, then run this skill to see which of those agents are active-and-dangerous at runtime.
References:
queries/cloud/agent365_observability.md — this skill references those queries rather than duplicating them. It contains the Plane A + Plane B equivalents, the RawEventData ↔ UnifiedAgentObservability field crosswalk, and the Defender-parity matrix.AlertInfo/AlertEvidence alert family (ServiceSource == "Security for AI": malicious URL, obfuscated/encoded payload, and other runtime-threat alerts) that is broader than, and complementary to, the Prompt Shield jailbreak/XPIA signal in §7.🔴 DETECT THE DATA PLANE FIRST. Probe UnifiedAgentObservability (Plane A) and CloudAppEvents (Plane B) before any analysis and proceed with whatever exists. Never assume a plane is present. See Data Plane Detection & Selection.
🔴 SAFETY VERDICTS ONLY COME FROM CloudAppEvents CopilotInteraction. UnifiedAgentObservability has no jailbreak/XPIA column. If CloudAppEvents is absent, state the safety gap explicitly.
🔴 If Plane A exists, run C0 — prompt/reply text is the highest-value evidence here. parse_json(EventOriginalRequestDetails).text returns empty on 100% of rows (no .text key); replies for Copilot Studio agents are on AISpanOutput. Never report a content gap without C0's coverage check.
ASK the user for scope and output format before generating:
reports/ai-agent-activity/) · both⛔ Evidence-based analysis only. Report ONLY what query results show. Use the explicit absence pattern (✅ No [finding] detected) for 0-result queries. Never invent agents, users, IPs, or counts.
🔴 NO composite /100 score. Surface the Risk Signal Catalog items, assign each a 🟢/🟡/🔴 verdict from the evidence, and show the reasoning.
🔴 DERIVE agent clusters from the actual inventory using the Agent Clustering Methodology. The example categories are illustrative only.
Timestamp column depends on the plane — UnifiedAgentObservability and CloudAppEvents in Data Lake use TimeGenerated; CloudAppEvents via Advanced Hunting uses Timestamp. See Known Pitfalls.
Enrich notable IPs (highest-volume agent egress IP, any jailbreak-source IP) with enrich_ips.py — parse the JSON via PowerShell, never read_file the .txt.
Report elapsed time after each phase.
The Agent 365 Observability SDK fans the same agent telemetry out to independent sinks. This skill uses the queryable planes and picks whichever the tenant has. (A fourth plane — Purview / DSPM for AI — carries the sensitive content of prompts; this skill points to it but does not build queries against it.)
| Plane | Table | Query tool | Carries | Retention |
|---|---|---|---|---|
| A · Sentinel Data Lake | UnifiedAgentObservability (workspaceId:"default") | mcp_sentinel-data_query_lake | Full span — prompt/reply text (extract with C0 — not .text), tool args, token usage, session graph | 90d+ |
| B · Microsoft Defender | CloudAppEvents (agent ActionTypes + CopilotInteraction) | RunAdvancedHuntingQuery (≤30d) or query_lake (workspace GUID, 90d) | Security metadata — agent/user/channel/tool names, ClientIP, jailbreak verdict. No prompt text / tool args / tokens. | ≤30d (AH) / 90d (Lake) |
C · CopilotActivity (supplemental, always worth probing) | CopilotActivity | RunAdvancedHuntingQuery (≤30d) or query_lake (90d) | Governance + runtime-protection signal that neither A nor B carry: full AppHost/RecordType surface breakdown (Security Copilot, Copilot Studio, Edge, SharePoint, M365AdminCenter, OutlookSidepane...), plugin/agent lifecycle events (CreateCopilotPlugin/EnableCopilotPlugin/DisableCopilotPlugin/DeleteCopilotPlugin/CopilotAgentManagement), and Defender Runtime Protection tool evaluations with FailClose posture (AccessedResources[].Type == "SecurityWebhook"). Validated empirically (2026-08-12): jailbreak detections and tool-call inventory in CopilotActivity are the same events as Plane B (no additional actors/hits, and Messages[].{Id,JailbreakDetected,isPrompt} carries no prompt text despite the table's name) — do not expect Plane C to add jailbreak-content depth. Its unique value is the fail-close/governance signal, which is exclusive to this table. | ≤30d (AH) / 90d (Lake) |
Run both probes (safe, cheap). Proceed based on which returns rows.
Probe A — Plane A present?
// mcp_sentinel-data_query_lake, workspaceId: "default"
UnifiedAgentObservability
| where TimeGenerated > ago(1d)
| summarize Rows = count()SemanticError: Failed to resolve table, the connector is not enabled → Plane A absent.)Probe B — Plane B present?
// RunAdvancedHuntingQuery (or query_lake with workspace GUID)
CloudAppEvents
| where Timestamp > ago(1d)
| where ActionType in ("InvokeAgent","InferenceCall","ExecuteToolBySDK","ExecuteToolByGateway","ExecuteToolByMCPServer","CopilotInteraction")
| summarize Rows = count()Probe C — Plane C (CopilotActivity) present?
// RunAdvancedHuntingQuery (or query_lake)
CopilotActivity
| where TimeGenerated > ago(1d)
| summarize Rows = count()| Plane A | Plane B | Decision |
|---|---|---|
| ✅ | ✅ | Plane A primary (richest — prompt text, tokens, tool args) + Plane B for the safety section (CopilotInteraction jailbreak verdict) and ClientIP enrichment. |
| ✅ | ❌ | Plane A only. Full activity/tool/token analysis. ⚠️ Safety section is limited — no jailbreak verdict without CloudAppEvents; state the gap. No ClientIP (UAO omits it). |
| ❌ | ✅ | Plane B only (the common Defender-only case). Metadata + ClientIP + safety verdicts. ⚠️ No prompt text, tool arguments, or token usage — content routes to Purview, not Defender; state the gap. |
| ❌ | ❌ | No agent telemetry. Report that neither plane is populated; suggest enabling the Agent 365 Observability connector (Plane A) and/or confirming Defender CloudAppEvents ingestion (Plane B). Stop. |
Plane C is additive, not a substitute — always layer it in when present (regardless of the A/B outcome), specifically for §7a below. It does not change the primary-plane decision above; it supplements whichever plane was selected with governance/runtime-protection signal neither A nor B carries.
Every report MUST open with a plane banner so the analyst knows what was and wasn't inspectable:
> 🛰️ Data plane: <A · Data Lake | B · Defender | A+B> [+ C · CopilotActivity governance].
> Prompt text / tool args / tokens: <available (Plane A) | not available (Plane B — content is in Purview)>.
> Safety verdicts (jailbreak/XPIA): <available via CloudAppEvents | UNAVAILABLE — CloudAppEvents not present>.
> ClientIP source enrichment: <available (Plane B) | not available (Plane A only)>.
> Governance/runtime-protection (Plane C): <available — CopilotActivity present | not available>.query_lake against the workspace GUID, not Advanced Hunting (AH silently truncates to 30d).AppHost/RecordType surface breakdown, plugin/agent lifecycle events, and Defender Runtime Protection tool FailClose posture.enrich_ips.py. Parse the JSON via PowerShell.Ask which scope at the start. All three share Phases 0–1; they differ in depth and filtering.
Full-fleet inventory + clustering + safety + risk signals. Uses the tenant-wide template.
Filter every query to one agent name. Include:
| where Source == "<agent>" or Target == "<agent>"); on Plane A check its blueprint family (C14). Draw the Agent → Agent handoff + tools diagram if any handoff exists.CopilotInteraction).
Uses the single-agent template.Filter to one UPN. Include:
Agent categories are environment-specific — derive them, don't impose them. Group the observed agents into a small number of clusters (typically 3–6) using any combination of these signals, then give each cluster a short descriptive name from what the data shows:
| Clustering signal | How to read it |
|---|---|
| Naming pattern | Shared prefixes/suffixes, versioned families (Finance Agent v2/v3), persona-named agents (agent named after a user's display name), *Test/*Demo build agents |
| Tool / connector set | Agents calling the same connectors cluster together (e.g. sentinelmcp:* → security-ops; GetDailyProcurementSnapshot → finance; a365outlookmailmcp + a365teamsmcp → personal-productivity) |
| Channel | Copilot Studio Test Pane / Evaluation → build/test; msteams / msteams:COPILOT → production/user-facing; Autonomous → background automation |
| User population | Single-user + single-IP + high volume → personal/background automation; many distinct users → shared or customer/supplier-facing |
| Owner / creator | A single builder iterating on a family of agents is a development pattern, not production traffic |
Illustrative example clusters (NOT a required taxonomy — name your own): Personal / Autonomous background agents, Security-Operations agents, Finance agents, Customer/Supplier-facing agents, Build/Test agents. Your report's clusters should reflect this tenant's data.
For each material cluster, produce: a short table (agents, primary users, channels, representative tools) + a user→agent→tool mermaid diagram (templates below).
Volume concentration is common and usually benign: one autonomous/background agent frequently dominates total tool-call volume (single user, single IP, steady 24/7 cadence). Call it out explicitly and separate it from the human-interactive long tail so it doesn't drown the analysis.
No composite /100 score. Evaluate each signal below against the query evidence, assign a 🟢/🟡/🔴 verdict, and show the reasoning. Only include signals the available plane can support (note gaps).
| # | Signal | Evidence source | 🔴 escalate when |
|---|---|---|---|
| 1 | Jailbreak / XPIA rate & clusters | CopilotInteraction JailbreakDetected / XPIA verdict, paired with the reply text from C0 | A flagged turn that produced a substantive reply (detected but not blocked), an adversarial prompt that was not flagged at all (coverage gap), partial compliance (refusal + payload), repeated hits on a customer/supplier-facing agent, a new agent/user pair, or hits followed by sensitive tool calls in-session |
| 2 | Sensitive-tool usage | Tool inventory — mail-send, data-write, directory-write, security-tooling (query_lake/Sentinel/SecurityCopilot), file-upload | Broadly-used or customer-facing agent invoking write/send/exfil-capable tools |
| 3 | Broadly-used / customer-facing runtime | Agent inventory (high distinct-user count) + external-facing channel | High-reach agent + sensitive tools + safety flags |
| 4 | New-tool first-seen | Tool baseline deviation (query library Q7a/7b) | An agent starts calling a tool absent from its prior baseline (scope drift / unauthorized addition) |
| 5 | Anomalous source IPs | ClientIP (Plane B) + enrich_ips.py | Genuine VPN/Tor/proxy with abuse reports on a non-Microsoft ISP. (Azure/Microsoft egress IPs are frequently vpnapi-flagged "VPN" with 0 abuse — a known FP; verify ISP + abuse score before escalating.) |
| 6 | Volume concentration | Agent inventory | A single agent dominating (>90%) fleet volume — usually benign automation; escalate only if the identity/IP/tool profile is unexpected |
| 7 | Tool-call failures / errors (Plane A only) | UAO EventErrorDetails (query library Q6) | A failure spike from a previously-stable agent (probing, broken MCP, permission revocation) |
| 8 | Runtime-protection fail-close posture (Plane C only) | CopilotActivity AccessedResources[].Type == "SecurityWebhook", extract FailClose | Any sensitive tool (mail-send, data-write, security-query) evaluated with FailClose = False — the agent proceeds even if the security evaluation can't complete |
| 9 | Plugin/agent lifecycle tampering (Plane C only) | CopilotActivity RecordType in (CreateCopilotPlugin, EnableCopilotPlugin, DisableCopilotPlugin, DeleteCopilotPlugin, CopilotAgentManagement) | Enable/create by an unexpected actor, a security-relevant plugin disabled, or high-volume CopilotAgentManagement by an unattributed (ActorName == "Unknown") system identity — confirm it's a known provisioning principal |
| 10 | Unexpected agent-to-agent handoff | Handoff edges from C13 (compound ConversationId) + blueprint families from C14 | A parent agent handing off to a sub-agent that is not part of its declared/known family, a handoff to a sensitive-tool-capable sub-agent, or a newly-appearing handoff edge absent from prior baseline (possible orchestration abuse / confused-deputy routing) |
Present these as a findings table with per-signal verdict, evidence, and a recommendation.
These are the compact, report-driving queries. The full validated set (Plane A 1a/4a/8a, Plane B 1b/4b/7b/8b, safety Query 9, session reconstruction 3a/3b, tool failures Q6) lives in
queries/cloud/agent365_observability.md— use those for anything beyond the basics.Timestamp column:
TimeGeneratedforUnifiedAgentObservabilityand forCloudAppEventsvia Data Lake;TimestampforCloudAppEventsvia Advanced Hunting. Queries below show theCloudAppEvents(Plane B) form usingTimestamp— swap toTimeGeneratedwhen running viaquery_lake.
🔴 parse_json(EventOriginalRequestDetails).text returns empty on 100% of rows — there is no .text key. The payload shape varies by agent hosting platform, and Copilot Studio agent replies are emitted on a separate AISpanOutput event, not on InvokeAgent. Use the shape-aware extractor below.
| Event type | Hosting platform | EventOriginalRequestDetails | EventOriginalResultDetails |
|---|---|---|---|
InvokeAgent | Copilot Studio | OTel array [{"role":"user","parts":[{"content":"…","type":"text"}]}] | empty — the reply is on the paired AISpanOutput row |
InvokeAgent | Foundry / Teams-hosted | bare text string (no JSON at all) | bare text string (the reply), or [] when the reply was suppressed |
AISpanOutput | Copilot Studio | empty | OTel array [{"finish_reason":"stop","role":"assistant","parts":[{"content":"…"}]}] — this is the agent reply |
ExecuteTool* | any | JSON args object, or key="value" string (SDK) | JSON object, or JSON-RPC array [{"jsonrpc":"2.0","result":{…}}] |
InferenceCall | Foundry | JSON array — full message history incl. the system prompt (can exceed 50 KB) | JSON array — model output |
let ReqText = (s:string) { case(
isempty(s), "",
s startswith "[", tostring(parse_json(s)[0].parts[0].content), // Copilot Studio OTel array
s startswith "{", "", // JSON object = tool args, not a message
s) }; // bare string = Foundry/Teams message
let ResText = (s:string) { case(
isempty(s), "",
s == "[]", "<<EMPTY REPLY — blocked/suppressed>>", // meaningful safety signal, not missing data
s startswith "[", tostring(parse_json(s)[0].parts[0].content),
s startswith "{", "",
s) };
UnifiedAgentObservability
| where TimeGenerated > ago(30d)
| where EventOriginalType in ("InvokeAgent","AISpanOutput")
| extend Agent = iff(isnotempty(SrcAgentName), SrcAgentName, tostring(TargetAgentName))
| extend Prompt = ReqText(EventOriginalRequestDetails),
Reply = ResText(EventOriginalResultDetails)
| where isnotempty(Prompt) or isnotempty(Reply)
| project TimeGenerated, EventOriginalType, Agent, ActorUsername, EventSessionId,
Prompt = substring(Prompt, 0, 1000), Reply = substring(Reply, 0, 1000), EventUid
| order by TimeGenerated ascReading the output: a Copilot Studio turn spans two rows — the
InvokeAgentrow carries the prompt, the followingAISpanOutputrow carries the reply. A Foundry/Teams turn is a singleInvokeAgentrow carrying both. Do not conclude "reply not captured" from an emptyInvokeAgentresult column without checking for a pairedAISpanOutput.Multi-part messages:
[0].parts[0]takes the first part of the first message, which covers ordinary user turns. For multi-part/multi-modal payloads,mv-expandthe array instead.Full text:
substring(...)truncates for readability — pull the untruncated payload byEventUidfor forensic review.
UnifiedAgentObservability
| where TimeGenerated > ago(30d)
| where isnotempty(EventOriginalRequestDetails) or isnotempty(EventOriginalResultDetails)
| extend ReqShape = case(EventOriginalRequestDetails startswith "[", "json-array",
EventOriginalRequestDetails startswith "{", "json-object",
isempty(EventOriginalRequestDetails), "empty", "bare-string"),
ResShape = case(EventOriginalResultDetails startswith "[", "json-array",
EventOriginalResultDetails startswith "{", "json-object",
isempty(EventOriginalResultDetails), "empty", "bare-string")
| summarize Rows = count(), Agents = make_set(iff(isnotempty(SrcAgentName), SrcAgentName, tostring(TargetAgentName)), 6)
by EventOriginalType, ReqShape, ResShape
| order by EventOriginalType asc, Rows descIf this returns rows, content exists — any failure to surface it is an extraction bug, not a telemetry gap. Only report a content gap when this query returns 0 rows or the shapes are all
empty.
A Prompt Shield JailbreakDetected = true verdict means the classifier fired, not that the request was blocked. Validated live: a system-prompt-extraction prompt was flagged true and the agent still returned its full system prompt, while an unflagged DAN-style prompt in the same session was suppressed ([] reply). Always pair the verdict with the reply text and report the outcome explicitly:
| Verdict | Reply | Report as |
|---|---|---|
true | <<EMPTY REPLY>> | 🟢 Detected and blocked |
true | substantive text | 🔴 Detected but NOT blocked — assess what leaked |
false | <<EMPTY REPLY>> | 🟡 Blocked by a different control — note the verdict gap |
false | substantive text, adversarial prompt | 🔴 Missed — Prompt Shield coverage gap |
Also inspect partial compliance: an agent that refuses the literal action ("I can't send email") but still produces the payload (a ready-to-paste draft containing the exfil address/URL) has not actually refused.
CloudAppEvents
| where Timestamp > ago(30d)
| where ActionType in ("InvokeAgent","InferenceCall","ExecuteToolBySDK","ExecuteToolByGateway","ExecuteToolByMCPServer","CopilotInteraction")
| summarize Events = count(), DistinctUsers = dcount(AccountId), FirstSeen = min(Timestamp), LastSeen = max(Timestamp) by ActionType
| order by Events descPlane A equivalent:
UnifiedAgentObservability | where TimeGenerated > ago(30d) | summarize Events=count() by EventOriginalType.
CloudAppEvents
| where Timestamp > ago(30d)
| where ActionType in ("InvokeAgent","InferenceCall","ExecuteToolBySDK","ExecuteToolByGateway","ExecuteToolByMCPServer")
| extend d = parse_json(RawEventData)
| extend AgentName = tostring(d.AgentName), TargetAgent = tostring(d.TargetAgentName)
| extend AgentName = iif(isnotempty(AgentName), AgentName, TargetAgent)
| summarize Events = count(),
UserPrompts = countif(ActionType == "InvokeAgent" and tostring(d.AgentBlueprintId) == "00000000-0000-0000-0000-000000000000"),
ToolCalls = countif(ActionType startswith "ExecuteTool"),
DistinctUsers = dcountif(tostring(d.UserId), tostring(d.UserId) != "N/A" and isnotempty(tostring(d.UserId))),
SourceIPs = dcount(tostring(d.ClientIP)),
Channels = make_set(tostring(d.ChannelName), 10),
FirstSeen = min(Timestamp), LastSeen = max(Timestamp)
by AgentName
| order by Events descPlane A equivalent: query library Query 1a (adds token usage + session-join agent-name attribution).
CloudAppEvents
| where Timestamp > ago(30d)
| where ActionType in ("ExecuteToolBySDK","ExecuteToolByGateway","ExecuteToolByMCPServer")
| extend d = parse_json(RawEventData)
| extend Agent = tostring(d.AgentName), ToolName = tostring(d.ToolName), ToolType = tostring(d.ToolType)
| where isnotempty(ToolName)
| summarize Calls = count(), Sessions = dcount(tostring(d.SessionIdentity)),
FirstCall = min(Timestamp), LastCall = max(Timestamp)
by Agent, ToolName, ToolType, ToolPath = ActionType
| order by Agent asc, Calls descPlane A equivalent: query library Query 4a.
CloudAppEvents
| where Timestamp > ago(30d)
| where ActionType in ("ExecuteToolBySDK","ExecuteToolByGateway","ExecuteToolByMCPServer")
| extend d = parse_json(RawEventData)
| where isnotempty(tostring(d.ToolName))
| summarize Calls = count(), Agents = dcount(tostring(d.AgentName)) by ToolType = tostring(d.ToolType)
| order by Calls descCloudAppEvents
| where Timestamp > ago(30d)
| where ActionType == "InvokeAgent"
| extend d = parse_json(RawEventData)
| where tostring(d.AgentBlueprintId) == "00000000-0000-0000-0000-000000000000" // user prompts only
| where tostring(d.UserId) != "N/A" and isnotempty(tostring(d.UserId))
| summarize Prompts = count(), Conversations = dcount(tostring(d.ConversationId)),
SourceIPs = dcount(tostring(d.ClientIP)), Agents = make_set(tostring(d.TargetAgentName), 10),
FirstPrompt = min(Timestamp), LastPrompt = max(Timestamp)
by Actor = tostring(d.UserId), Channel = tostring(d.ChannelName)
| order by Prompts descPlane A equivalent: query library Query 8a.
CloudAppEvents
| where Timestamp > ago(30d)
| where ActionType in ("InvokeAgent","InferenceCall","ExecuteToolBySDK","ExecuteToolByGateway","ExecuteToolByMCPServer")
| summarize Events = count(), Users = dcount(AccountId) by bin(Timestamp, 1d)
| order by Timestamp ascCloudAppEvents
| where Timestamp > ago(30d)
| where ActionType == "CopilotInteraction"
| where RawEventData has_any ("JailbreakDetected","jailbreakDetected","xpiaDetected","indirectPromptInjection","classifications")
| extend P = parse_json(RawEventData)
| mv-expand Msg = P.CopilotEventData.Messages
| extend Jb = tobool(coalesce(Msg.JailbreakDetected, Msg.jailbreakDetected)),
Xpia = tobool(coalesce(Msg.xpiaDetected, Msg.indirectPromptInjectionDetected))
| where Jb == true or Xpia == true
| extend AgentName = tostring(coalesce(P.AgentName, P.CopilotEventData.TargetAgentName)),
UserUpn = tostring(P.UserId), AppHost = tostring(P.CopilotEventData.AppHost)
| summarize Hits = count(), Users = dcount(UserUpn), FirstSeen = min(Timestamp), LastSeen = max(Timestamp)
by AgentName, AppHost, Verdict = case(Jb, "jailbreak", Xpia, "xpia", "other")
| order by Hits desc
JailbreakDetectedis PascalCase in current tenants. Drill a cluster by adding| where tostring(P.UserId) =~ "<upn>"and projectingTimeGenerated, IPAddress, AgentName, ThreadId=tostring(P.CopilotEventData.ThreadId).
Detects sub-agent handoffs — a parent agent routing a turn to a sub-agent — plus every agent→tool edge, in one edge list ready for the Agent → Agent handoff + tools diagram. There is no dedicated "Agent A invoked Agent B" event: a handoff is revealed by a compound ConversationId on the sub-agent's InvokeAgent rows — the parent's own ConversationId, then _, then a new child GUID (<parentConversationId>_<childConversationId>). Safe to split on _ because conversation GUIDs use only hyphens.
let Lookback = 30d;
let Raw =
CloudAppEvents
| where Timestamp > ago(Lookback)
| where ActionType in ("InvokeAgent","ExecuteToolBySDK","ExecuteToolByGateway","ExecuteToolByMCPServer") // most selective filter first
| extend d = parse_json(RawEventData) // parse once
| extend SrcAgent = tostring(d.AgentName), DstAgent = tostring(d.TargetAgentName),
ToolName = tostring(d.ToolName), ToolType = tostring(d.ToolType),
ConvId = tostring(d.ConversationId), SessionId = tostring(d.SessionIdentity)
| extend ConvParts = split(ConvId, "_")
| extend RootConvId = tostring(ConvParts[0]), IsSubThread = array_length(ConvParts) > 1;
let ThreadOwner = Raw
| where ActionType == "InvokeAgent" and isnotempty(DstAgent)
| summarize OwnerAgent = take_any(DstAgent) by ConvId;
let AgentToTool = Raw
| where ActionType in ("ExecuteToolBySDK","ExecuteToolByGateway","ExecuteToolByMCPServer")
| where isnotempty(SrcAgent)
| summarize Count = count(), Sessions = dcount(SessionId), FirstSeen = min(Timestamp), LastSeen = max(Timestamp)
by Source = SrcAgent, EdgeType = "Agent -> Tool", Target = ToolName, Detail = ToolType;
let AgentToAgent = Raw
| where ActionType == "InvokeAgent" and IsSubThread
| join kind=leftouter (ThreadOwner | project RootConvId = ConvId, ParentAgent = OwnerAgent) on RootConvId
| where isnotempty(ParentAgent) and ParentAgent != DstAgent
| summarize Count = count(), Sessions = dcount(SessionId), FirstSeen = min(Timestamp), LastSeen = max(Timestamp)
by Source = ParentAgent, EdgeType = "Agent -> Agent (handoff)", Target = DstAgent, Detail = "sub-agent conversation";
union AgentToTool, AgentToAgent
| project Source, EdgeType, Target, Detail, Count, Sessions, FirstSeen, LastSeen
| order by Source asc, EdgeType asc, Count descPlane A equivalent: query library Query 10b (same edge shape, simpler —
SrcAgentName/TargetAgentName/ToolNameare typed columns, noRawEventDataparsing). Scope to one agent by appending| where Source == "<Agent>" or Target == "<Agent>".
Groups agent identities that share a SrcAgentBlueprintId — a direction-agnostic signal that a parent orchestrator and its sub-agent(s) belong to the same deployed multi-agent solution. CloudAppEvents never populates the sub-agent's identity, so this cross-check is Data-Lake-only. Use it as a fast tenant-wide sweep to find handoff candidates, then run C13 for direction + counts.
// mcp_sentinel-data_query_lake, workspaceId: "default"
UnifiedAgentObservability
| where TimeGenerated > ago(30d)
| where isnotempty(SrcAgentName) and isnotempty(SrcAgentBlueprintId) and SrcAgentBlueprintId != "00000000-0000-0000-0000-000000000000"
| summarize Agents = make_set(SrcAgentName), AgentIds = make_set(SrcAgentId), Events = count() by SrcAgentBlueprintId
| extend AgentCount = array_length(Agents)
| where AgentCount > 1
| order by AgentCount descA row with
AgentCount > 1is a candidate multi-agent solution. Keep the zero-GUIDSrcAgentBlueprintIdexclusion — M365 Copilot built-ins all share it and would otherwise collapse into one false-positive "family."
| where tostring(d.AgentName) =~ "<agent>" (or tostring(P.AgentName) / tostring(P.CopilotEventData.TargetAgentName) for safety) to C2–C7.| where tostring(d.UserId) =~ "<upn>" (or tostring(P.UserId) for safety).CopilotActivity governance & runtime-protectionRun these whenever Probe C returns rows, in addition to whichever of Plane A/B was selected as primary. They surface signal that neither Plane A nor Plane B carries. Full query set and pitfalls: queries/cloud/copilot_activity_investigation.md.
CopilotActivity
| where TimeGenerated > ago(30d)
| summarize Events = count(), Actors = dcount(ActorName), Agents = dcountif(AgentId, isnotempty(AgentId)),
FirstSeen = min(TimeGenerated), LastSeen = max(TimeGenerated)
by RecordType, AppHost
| order by Events descReveals surfaces the Plane B
ActionTypefilter misses entirely — Security Copilot (AppHostprefixedSecurityCopilot-<guid>, frequently the single largest volume source in the tenant), Edge, SharePoint, M365AdminCenter, OutlookSidepane, Copilot Studio,pva-maker-evaluation. Use this to sanity-check that the Plane B agent inventory isn't missing an entire surface.
CopilotActivity
| where TimeGenerated > ago(30d)
| where RecordType == "CopilotInteraction"
| extend AR = parse_json(tostring(LLMEventData.AccessedResources))
| mv-expand AR
| where tostring(AR.Type) == "SecurityWebhook"
| extend EvalText = tostring(AR.Action)
| extend ToolName = extract(@"Evaluated tool name: ([^,]+)", 1, EvalText),
FailClose = extract(@"Fail close configuration is set to: (\w+)", 1, EvalText)
| summarize Evaluations = count() by ToolName, FailClose
| order by Evaluations descHigh-value, Plane-C-exclusive signal. Flag any sensitive tool name (mail-send, data-write, security-query connectors) with
FailClose = False— the agent proceeds even if the security evaluation can't complete. This has no equivalent in Plane A or Plane B.
CopilotActivity
| where TimeGenerated > ago(30d)
| where RecordType in ("CreateCopilotPlugin","UpdateCopilotPlugin","EnableCopilotPlugin","DisableCopilotPlugin","DeleteCopilotPlugin","CopilotAgentManagement")
| summarize Events = count(), Actors = dcount(ActorName), TopActors = make_set(ActorName, 5) by RecordType
| order by Events descGovernance/tampering signal absent from Plane A/B — neither table logs plugin enable/disable/create/delete events. Watch for
ActorName == "Unknown"on high-volumeCopilotAgentManagement— confirm it's a known provisioning/system principal, not an unattributed actor.
CopilotActivity
| where TimeGenerated > ago(30d)
| where isnotempty(AgentId)
| extend AR = parse_json(tostring(LLMEventData.AccessedResources))
| mv-expand AR
| extend SiteUrl = tostring(AR.SiteUrl), ResourceType = tostring(AR.Type)
| where SiteUrl has ".sharepoint.com"
| summarize AccessCount = count(), Users = dcount(ActorName), Sites = make_set(SiteUrl, 20)
by AgentName, AgentId
| order by AccessCount descOnly meaningful for agents that actually read SharePoint/OneDrive content — returns 0 rows for agents with no tool/knowledge-source calls (confirmed empirically: an agent showing 0 tool calls in Plane B also shows 0
AccessedResourceshere — the two are consistent, not contradictory).
Validated NOT additive (validated 2026-08-12): the CopilotActivity jailbreak query (Messages[].JailbreakDetected) and its tool-call inventory (AppHost == "Autonomous" → AccessedResources[].Type == "Connector") return the same events, same actors, same counts as their Plane B equivalents (C7 and C3). Messages carries only {Id, JailbreakDetected, isPrompt} — no prompt text, despite what the table name might suggest. Don't spend time re-deriving §7 (Safety Layer) or §6 (Tool Usage) from Plane C if Plane B is already primary — use Plane C specifically for C8–C11 instead.
Microsoft Defender's Security for AI capability (integrated with Agent 365) generates its own native alerts/incidents for AI agent runtime threats — malicious URL submission, obfuscated/encoded/hidden payloads, secret leakage, LLM recon, suspicious IP/user access — distinct from (and broader than) the Prompt Shield jailbreak/XPIA boolean in C7. When enabled, these surface via AlertInfo/AlertEvidence with ServiceSource == "Security for AI" (Advanced Hunting only — not queryable in Sentinel Data Lake). See Detect and investigate threats to AI agents using Microsoft Defender (Preview).
Identify — is it enabled, and what's been flagged? (30d)
AlertInfo
| where TimeGenerated > ago(30d)
| where ServiceSource == "Security for AI"
| summarize Alerts = count(), FirstSeen = min(TimeGenerated), LastSeen = max(TimeGenerated)
by Title, Category, Severity
| order by Alerts descZero rows means either no findings this window, or the feature isn't enabled — confirm in Defender portal → Settings → Security for AI before concluding "clean."
Gather context — full evidence per alert (user, agent, hosting platform, related URL/IP)
AlertEvidence
| where TimeGenerated > ago(30d)
| where ServiceSource == "Security for AI"
| extend AF = parse_json(AdditionalFields)
| summarize
Title = any(Title), Category = any(Category), Severity = any(Severity),
Users = make_set_if(AccountUpn, EntityType == "User" and isnotempty(AccountUpn)),
Agents = make_set_if(tostring(AF.AgentName), EntityType == "AIAgent"),
HostingPlatforms = make_set_if(tostring(AF.HostingPlatformType), EntityType == "AIAgent"),
RelatedUrls = make_set_if(RemoteUrl, EntityType == "Url" and isnotempty(RemoteUrl)),
RelatedIPs = make_set_if(RemoteIP, EntityType == "Ip" and isnotempty(RemoteIP))
by AlertId, AlertTime = TimeGenerated
| order by AlertTime descAdd
| where AlertId == "<AlertId>"to drill into one specific alert.AIAgent/URL/IP detail lives inAdditionalFields(JSON) — alwaysparse_json()before extracting.
Resolve to a reportable Incident ID. AlertInfo/AlertEvidence carry no IncidentId column — for the §7a report table (and any output shown to the user), resolve each AlertId to its Defender XDR incident via GetAlertById(alertId="<AlertId>") (Triage MCP), which returns incidentId directly. Multiple AlertIds frequently share the same incidentId (they're correlated into one multi-stage incident) — dedupe before presenting. Never show a bare AlertId in a report; always render [#<IncidentId>](https://security.microsoft.com/incidents/<IncidentId>?tid=<tenant_id>) per the SecurityIncident Query & Output Standards global rule.
Known pitfalls:
ProductName doesn't exist on AlertInfo/AlertEvidence — use ServiceSource/DetectionSource instead (both literal "Security for AI").SecurityAlert/SecurityIncident.AlertIds (validated on a live multi-stage incident: 2 of 4 correlated alerts were native Security-for-AI alerts, and neither resolved via a SecurityAlert.SystemAlertId lookup). For incident-level pivoting, use GetIncidentById(incidentId="<ProviderIncidentId>", includeAlertsData=true) (Triage MCP) instead of a Sentinel-side join.BehaviorInfo (ActionType == "BehaviorPromptShieldJailbreakDetect") undercounts jailbreak hits relative to the turn-level C7 query — validated 6 vs. 17 hits in the same 30-day window. Treat C7 as authoritative for jailbreak counts; use BehaviorInfo only as a supplementary correlation signal, not for trend/volume reporting.RunAdvancedHuntingQuery.Use these to visualize the derived clusters and volume. Substitute real names/counts from query results.
xychart-beta
title "Agent Events per Day"
x-axis [<day labels>]
y-axis "Events" 0 --> <max>
bar [<daily counts>]flowchart LR
U1(["user@contoso.com"])
A1["<Agent Name><br/>(cluster: <cluster>)"]
T1["<tool / connector>"]
U1 -->|"<prompts> prompts · <channel>"| A1
A1 -->|"<calls>"| T1
classDef user fill:#1e3a5f,stroke:#3b82f6,color:#fff;
classDef agent fill:#14532d,stroke:#22c55e,color:#fff;
classDef tool fill:#3f3f46,stroke:#a1a1aa,color:#fff;
class U1 user;
class A1 agent;
class T1 tool;Use when C13/C14 surface a sub-agent handoff. Shows both the agent→tool edges and the parent→sub-agent handoff (==>) in one topology. Draw one per multi-agent family. Edge labels carry the handoff turn count / tool call count from the C13 edge list.
flowchart LR
U(["user@contoso.com"])
P["<Parent / Orchestrator Agent>"]
S["<Sub-Agent><br/>(handed-off task)"]
T1["<parent tool / connector>"]
T2["<sub-agent tool / connector>"]
U -->|"<prompts> prompts · <channel>"| P
P -->|"<calls>"| T1
P ==>|"handoff ×<turns><br/>(sub-agent conversation)"| S
S -->|"<calls>"| T2
classDef user fill:#1e3a5f,stroke:#3b82f6,color:#fff;
classDef agent fill:#14532d,stroke:#22c55e,color:#fff;
classDef subagent fill:#134e4a,stroke:#2dd4bf,color:#fff;
classDef tool fill:#3f3f46,stroke:#a1a1aa,color:#fff;
class U user;
class P agent;
class S subagent;
class T1,T2 tool;A sub-agent that makes no tool calls (e.g. a conversational logger) simply has no outgoing
-->edge — that is expected, not missing data. The handoff==>edge alone is the relationship.
flowchart LR
U(["user@contoso.com"])
IP(["<ip> — <enrichment verdict>"])
A["<Agent Name>"]
PS{{"Prompt Shield<br/>JailbreakDetected ×<N>"}}
OUT["<in-session tool escalation? yes/no>"]
IP -->|"session"| U
U -->|"<N> prompts"| A
A -->|"safety inspection"| PS
PS --> OUT
classDef user fill:#1e3a5f,stroke:#3b82f6,color:#fff;
classDef agent fill:#14532d,stroke:#22c55e,color:#fff;
classDef flag fill:#78350f,stroke:#f59e0b,color:#fff;
class U user; class A agent; class PS,OUT flag;Ask before generating:
reports/ai-agent-activity/:Agent_Activity_Report_Tenant_<org>_<YYYY-MM-DD>.mdAgent_Activity_Report_Agent_<agent-slug>_<YYYY-MM-DD>.mdAgent_Activity_Report_User_<upn-slug>_<YYYY-MM-DD>.mdAll templates open with the mandatory plane banner. Omit sections the active plane cannot support, and state why (gap note).
Every report ends with a Suggested Follow-Up Prompts section — copy-paste-ready prompts that let the analyst drill from the fleet view into a specific agent, user, session, or turn without having to know the skill's scope syntax.
Rules:
ai-agent-posture for configuration, user-investigation for identity context, incident-investigation for a correlated incident ID.Prompt patterns to draw from:
| Goal | Prompt shape |
|---|---|
| All activity for one agent | Run an AI agent activity report for agent "<Agent Name>", last <N> days |
| All activity for one user | Run an AI agent activity report for user <upn>, last <N> days |
| One user ↔ one agent | Show every interaction between <upn> and agent "<Agent Name>" over the last <N> days, with prompts and replies |
| Full transcript of a session | Reconstruct session <EventSessionId> — prompts, tool calls, and replies in order |
| A specific flagged turn | Show the prompt and the agent's reply for the <verdict> hit on <Agent Name> at <timestamp> |
| Tool arguments for an agent | Show every tool call "<Agent Name>" made with its arguments and results, last <N> days |
| Config for a flagged agent | Run an AI agent posture audit for agent "<Agent Name>" |
| Identity context for a user | Investigate user <upn> |
| Correlated incident | Investigate incident <ProviderIncidentId> |
# AI Agent Activity Report — <Tenant / Org>
**Report window:** <start> → <end> (<N> days)
**Data source:** <plane(s) used> · <table(s)>
**Report generated:** <date>
> 🛰️ Data plane: <A | B | A+B>. Prompt text/tool args/tokens: <avail/gap>. Safety verdicts: <avail/gap>. ClientIP enrichment: <avail/gap>.
## 1. Executive Summary
<2–4 sentences: total events, agent count, dominant workload, safety posture, notable/new findings>
## 2. Scope & Methodology
<which plane, why, what it can/can't see (gap notes)>
## 3. Volume (window)
| ActionType | Events | Distinct Users | First → Last |
|---|---:|---:|---|
### Daily trend
<xychart-beta>
## 4. Agent Inventory
<top-N table: agent, events, users, channels; note the long tail>
## 5. Agent Clusters
<for each derived cluster: short table + user→agent→tool mermaid>
## 6. Tool / Connector Usage
<tool-type distribution table + notable sensitive-tool usage>
## 7. Safety Layer — Jailbreak / XPIA <or: "not available — no CloudAppEvents">
<hits by agent/user/channel + most-recent-cluster drill-down + safety mermaid + IP enrichment>
### 7a. Security for AI native alerts <omit if Probe 0 in C12 returns 0 rows>
| Incident | Title | Category | Severity | Time | User | Agent | Hosting Platform | Related URL | Related IP |
|---|---|---|---|---|---|---|---|---|---|
<one row per alert — **Incident column is a clickable link**, never a bare AlertId: `[#<IncidentId>](https://security.microsoft.com/incidents/<IncidentId>?tid=<tenant_id>)`. Resolve each alert's `incidentId` via `GetAlertById` (Triage MCP) — `AlertInfo`/`AlertEvidence` carry no `IncidentId` column. Read `tenant_id` from `config.json`; omit `?tid=` if not configured.>
## 8. Risk Signals
| # | Signal | Verdict | Evidence | Recommendation |
|---|--------|:------:|----------|----------------|
## 9. Findings & Recommendations
<prioritized, evidence-based>
## 10. Suggested Follow-Up Prompts
<5–7 copy-paste prompts derived from THIS report's results — see [guidance](#follow-up-prompt-guidance-applies-to-every-template). Order by evidence, not volume.>
**Drill into a specific agent**
- `Run an AI agent activity report for agent "<top/flagged agent>", last <N> days` — <why: e.g. highest tool-call volume; carries the only write-capable connector>
**Drill into a specific user**
- `Run an AI agent activity report for user <upn>, last <N> days` — <why>
**Drill into one user↔agent pair**
- `Show every interaction between <upn> and agent "<Agent Name>" over the last <N> days, with prompts and replies` — <why>
**Drill into a flagged turn or session**
- `Reconstruct session <EventSessionId> — prompts, tool calls, and replies in order` — <why>
- `Show the prompt and the agent's reply for the <verdict> hit on <Agent Name> at <timestamp>` — <why>
**Hand off to another skill**
- `Run an AI agent posture audit for agent "<Agent Name>"` — <why: config-side view of a runtime finding>
- `Investigate incident <ProviderIncidentId>` — <why: correlated Defender incident>
## 11. Appendix — Queries Used
<the KQL run, with the plane/timestamp variant noted># AI Agent Activity Report — <Agent Name>
**Agent:** <name> · **Report window:** <start> → <end> · **Data source:** <plane> · **Generated:** <date>
> 🛰️ <plane banner>
## 1. Summary
<events, distinct users, channels, first/last seen, safety verdict>
## 2. Users & Channels
<who used it, over which channels, source IPs (Plane B)>
## 3. Tools Invoked
<per-tool call counts + tool types>
## 4. Session Reconstruction
<Plane A: prompt→tool→reply timeline (query library 3a/3b) · Plane B: metadata timeline (no content)>
## 5. Safety Flags
<CopilotInteraction hits for this agent, if any>
## 6. Risk Signals & Recommendations
## 7. Suggested Follow-Up Prompts
<derived from this agent's actual users, sessions, and tools — see [guidance](#follow-up-prompt-guidance-applies-to-every-template)>
- `Run an AI agent activity report for user <upn>, last <N> days` — <why: this agent's heaviest / only flagged user>
- `Show every interaction between <upn> and agent "<this agent>" over the last <N> days, with prompts and replies` — <why>
- `Reconstruct session <EventSessionId> — prompts, tool calls, and replies in order` — <why: longest / flagged session>
- `Show every tool call "<this agent>" made with its arguments and results, last <N> days` — <why: sensitive or newly-seen tool>
- `Run an AI agent posture audit for agent "<this agent>"` — <why: confirm declared tools/access match observed runtime># AI Agent Activity Report — <user@contoso.com>
**User:** <upn> · **Report window:** <start> → <end> · **Data source:** <plane> · **Generated:** <date>
> 🛰️ <plane banner>
## 1. Summary
<agents used, total prompts, channels, source IPs, safety verdict>
## 2. Agents Used
<agent, prompts, channels, first/last seen>
## 3. Tools Run On Behalf
<tools invoked across the user's agent sessions>
## 4. Safety Flags
<CopilotInteraction hits attributed to this user + IP enrichment>
## 5. Risk Signals & Recommendations
## 6. Suggested Follow-Up Prompts
<derived from this user's actual agents, sessions, and flags — see [guidance](#follow-up-prompt-guidance-applies-to-every-template)>
- `Run an AI agent activity report for agent "<Agent Name>", last <N> days` — <why: agent this user hit hardest / where the flag landed>
- `Show every interaction between <this user> and agent "<Agent Name>" over the last <N> days, with prompts and replies` — <why>
- `Reconstruct session <EventSessionId> — prompts, tool calls, and replies in order` — <why>
- `Show the prompt and the agent's reply for the <verdict> hit on <Agent Name> at <timestamp>` — <why>
- `Investigate user <this user>` — <why: sign-in / identity context for the agent activity>| Pitfall | Detail / Fix |
|---|---|
🔴 parse_json(EventOriginalRequestDetails).text returns empty — there is no .text key | Validated live: .text recovered 0 of 160 content rows while the shape-aware extractor recovered 104/105 prompts and 55/55 replies. The payload is a bare text string (Foundry/Teams hosts) or an OTel message array [{"role":…,"parts":[{"content":…}]}] (Copilot Studio hosts) — never {"text":…}. Fix: use C0. Never conclude "prompt text unavailable" from an empty .text result. |
🔴 Copilot Studio agent replies are on AISpanOutput, not InvokeAgent | For Copilot Studio agents, InvokeAgent.EventOriginalResultDetails is empty on every row — the reply is emitted as a separate AISpanOutput event with the text in EventOriginalResultDetails. Concluding "agent reply not captured" from the InvokeAgent row alone is wrong. Foundry/Teams-hosted agents do put the reply on the InvokeAgent row. Always include AISpanOutput when reconstructing a conversation. |
AISpanOutput is undocumented but carries real content | It does not appear in most EventOriginalType reference tables, yet it accounted for ~27% of Plane A rows in a validated tenant and is the sole source of Copilot Studio reply text. Include it in inventory, volume, and transcript queries. |
[] in a result column is a safety signal, not missing data | An empty JSON array as EventOriginalResultDetails means the reply was suppressed/blocked. Report it as a blocked turn, not as a telemetry gap. |
JailbreakDetected = true does not mean the request was blocked | The verdict records that the classifier fired, not that the model refused. Validated live: a flagged system-prompt-extraction turn still returned the full system prompt, while an unflagged DAN-style turn was suppressed. Always cross-check the reply — see Detection ≠ prevention. |
| Refusal text can still contain the payload | "I can't send emails… but here's a draft" followed by the exfil address and URL is partial compliance, not a refusal. Read the whole reply before scoring the turn as blocked. |
InferenceCall request payloads can exceed 50 KB | They carry the full message history including the system prompt. Useful for confirming a system-prompt leak, but substring() them — don't project raw. |
| Assuming a data plane exists | Always run Probe A + Probe B first. UnifiedAgentObservability absent → Plane B; CloudAppEvents absent → Plane A only (safety gap). |
Timestamp vs TimeGenerated | UnifiedAgentObservability → TimeGenerated. CloudAppEvents via Advanced Hunting → Timestamp; via Data Lake query_lake → TimeGenerated. Wrong column returns 0 rows or SemanticError. |
workspaceId:"default" for Plane A | UnifiedAgentObservability is a Data Lake system table — query with workspaceId:"default", not a workspace GUID (GUID returns table-not-found). |
| Safety verdict only in CloudAppEvents | UnifiedAgentObservability has no jailbreak/XPIA column. Safety section always uses CloudAppEvents CopilotInteraction. |
JailbreakDetected is PascalCase | In current tenants the key is JailbreakDetected (not jailbreakDetected). xpiaDetected/classifications/prompt text are often absent — coalesce() and rely on JailbreakDetected. |
ExecuteToolByGateway has no ClientIP | Gateway/CodefulServer/RemoteMCP tool rows leave ClientIP blank — source-IP enrichment is only reliable on InvokeAgent/CopilotInteraction rows. |
| 30-day AH cap | RunAdvancedHuntingQuery silently truncates to 30d. For >30d on Plane B, use query_lake against the workspace GUID (TimeGenerated). |
Plane A emits ~2× InvokeAgent | Plane A logs agent replies as separate InvokeAgent rows; Plane B InvokeAgent is almost all user prompts. Discriminate with AgentBlueprintId / SrcAgentBlueprintId (zero-GUID = user prompt). Don't compare raw InvokeAgent counts across planes. |
AccountId/UserId GUID vs UPN | On CloudAppEvents, RawEventData.UserId is the UPN; AccountId is the GUID. Filter users by tostring(d.UserId). |
RawEventData is a large JSON blob | Parse once (extend d = parse_json(RawEventData)) then read d.<field>; never tostring(RawEventData) has "x" for filtering. |
| No prompt content on Plane B | Prompt text, tool arguments, and token usage are not in CloudAppEvents (they route to Purview). For content, use Plane A via C0 — or point the analyst to Purview / DSPM for AI only if Plane A is genuinely absent or its content columns are genuinely empty (verify with the C0 coverage self-check first). |
| Azure-IP "VPN" false positive | enrich_ips.py (vpnapi.io) frequently flags Microsoft/Azure egress IPs as "VPN" with 0 abuse reports — a known FP. Verify ISP + abuse score before treating an IP as anomalous. |
CopilotActivity (Plane C) name suggests prompt content — it doesn't have any | LLMEventData.Messages[] carries only {Id, JailbreakDetected, isPrompt} — despite the table's "LLM" naming, there is no prompt/reply text. Content still routes to Purview/DSPM for AI regardless of which plane you query. |
CopilotActivity jailbreak/tool-call queries duplicate Plane B, not extend it | Validated empirically: the same jailbreak hits and Autonomous-agent tool calls appear in both tables with identical counts. Use Plane C for C8–C11 (surface breakdown, runtime-protection fail-close, plugin lifecycle, SharePoint access) — not to re-derive §6/§7 already covered by Plane B. |
Microsoft Learn samples reference LLMActivity | The actual table name in Advanced Hunting and the Sentinel workspace is CopilotActivity. LLMActivity returns Failed to resolve table. |
CopilotActivity uses TimeGenerated, not Timestamp, in both AH and Data Lake | Unlike CloudAppEvents (which needs Timestamp in AH), CopilotActivity is consistent — always TimeGenerated. |
Security for AI native alerts don't reliably sync to Sentinel SecurityIncident/SecurityAlert | Validated on a live multi-stage incident: 2 of 4 correlated alerts were native ServiceSource == "Security for AI" alerts (AlertId prefixed ai...) — neither resolved via a Sentinel SecurityAlert.SystemAlertId lookup. Use GetIncidentById(includeAlertsData=true) or C12 above — not a Sentinel-side join — to pivot into these alerts. |
BehaviorInfo jailbreak rows are a subset of the turn-level signal | BehaviorInfo (ActionType == "BehaviorPromptShieldJailbreakDetect") undercounts relative to CloudAppEvents/CopilotActivity Messages[].JailbreakDetected (validated: 6 vs. 17 hits, same 30d window). Treat CloudAppEvents/CopilotActivity as authoritative for jailbreak counts; use BehaviorInfo only as a supplementary agent/user correlation signal. |
| Agent-to-agent handoffs have no dedicated event | Neither plane emits an "Agent A invoked Agent B" event, and AgentsInfo config doesn't declare sub-agents as callable actions — a handoff is a Copilot Studio runtime routing decision. Detect it via the compound ConversationId (<parent>_<child>) on the sub-agent's InvokeAgent rows (C13). The parent keeps emitting its own root-ConversationId rows throughout — it stays the orchestrating shell. In CloudAppEvents the sub-agent's AgentName/AgentId are blank (both look like ordinary user-prompt rows — only the ConversationId structure reveals the handoff); in UnifiedAgentObservability the sub-agent gets a real SrcAgentId/SrcAgentName on its AISpanOutput rows and shares the parent's SrcAgentBlueprintId (C14). |
AISpanOutput (Copilot Studio reply text) as well as InvokeAgentTimeGenerated vs Timestamp)CloudAppEvents CopilotInteraction; if absent, the gap is stated explicitlyenrich_ips.py (JSON parsed, not .txt read); Azure-VPN FP considered✅ No … detected)reports/ai-agent-activity/ with the correct scope filename© SCStelz, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
Just SKILL.md in .github/skills/ai-agent-activity of SCStelz/security-investigator.
Open the folder on GitHubat commit 51e1385
AI Agent Activity next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| AI Agent Activity this skillSCStelz/security-investigator | 249 | — | ~17k | Automated safety check: Pass | MIT | |
| Security GuidejnMetaCode/shellward | 140 | — | ~644 | Automated safety check: Warn | Apache-2.0 | |
| China AI Compliance AuditjnMetaCode/shellward | 140 | — | ~1.1k | Automated safety check: Pass | Apache-2.0 | |
| Prompt Injection Defensesickn33/agentic-awesome-skills | 47k | 2 repos | ~4.2k | Automated safety check: Warn | MIT | |
| Moai Ref LLM Securitymodu-ai/moai-adk | 1.2k | — | ~4.5k | Automated safety check: Pass | Apache-2.0 | |
| Detecting Indirect Prompt Injectionmukul975/Anthropic-Cybersecurity-Skills | 34k | — | ~2.8k | Automated safety check: Warn | Apache-2.0 |
jnMetaCode/shellward
OpenClaw 安全部署指南 / Security deployment guide — help users secure their OpenClaw installation
jnMetaCode/shellward
按中国法规(网安法 / PIPL / 等保2.0 / 数据出境 / AI生成内容标识)审计一个 AI 项目的代码仓库,产出每条都带 文件:行 取证、经独立复核、经脚本校验的合规报告。当用户问「这个项目上线合不合规」「调用了 OpenAI/Claude 算不算数据出境」「要不要做 AI 标识」「帮我做合规自查/等保/PIPL 检查」时使用。Audit an AI project's…
sickn33/agentic-awesome-skills
Defend AI systems against prompt injection and indirect prompt attacks using input controls, tool permissions, output validation, and isolation boundaries.
modu-ai/moai-adk
AI/LLM defensive security reference: prompt-injection defense, OWASP LLM Top 10 defensive mapping, MCP and agentic tool-call hardening, training-data poisoning detection, model-output validation and…
mukul975/Anthropic-Cybersecurity-Skills
Detect and defend against indirect prompt injection hidden in web pages, documents, and images consumed by an agent, via content extraction (HTML/PDF/OCR), normalization, and scanning with LLM…
mukul975/Anthropic-Cybersecurity-Skills
Runs NVIDIA garak probe suites (jailbreak, prompt injection, data leakage, toxicity, and more) against an LLM endpoint - Hugging Face models, OpenAI-compatible APIs, or Bedrock - then interprets the…
SCStelz/security-investigator
A skill your agent uses when asked to investigate Conditional Access policy changes, sign-in failures related to CA policies (error codes 53000, 50074, 530032), or suspected policy…
SCStelz/security-investigator
Weekly review of an investigation tenant-context memory file against the most recent SOC scan reports (e.g.
SCStelz/security-investigator
A skill your agent uses when asked to create heatmaps, visualize patterns over time, show activity grids, or display aggregated data in a matrix format.
SCStelz/security-investigator
Audit or report on AI agent security posture across Copilot Studio, Microsoft 365 Copilot, Microsoft Foundry, and third-party agents.
SCStelz/security-investigator
Audit Entra ID app registration and service principal security posture.
SCStelz/security-investigator
A skill your agent uses when asked to trace authentication flows, analyze SessionId chains, investigate token reuse vs interactive MFA, or assess geographic anomalies in sign-ins.
Works with
Categories
Report/investigate RUNTIME ACTIVITY of AI agents (Agent 365 / Copilot Studio / M365 Copilot / Work IQ) — agents used, tools/connectors, channels, tokens, prompt/reply content, and Prompt Shield…. AI Agent Activity is an agent skill from SCStelz/security-investigator. Report/investigate RUNTIME ACTIVITY of AI agents (Agent 365 / Copilot Studio / M365 Copilot / Work IQ) — agents used, tools/connectors, channels, tokens, prompt/reply content, and Prompt Shield jailbreak/XPIA verdicts.
AI Agent Activity fits situations like: tasks that involve LLM guardrails; tasks that involve Prompt injection and agent security.
Run `npx skills add SCStelz/security-investigator --skill ai-agent-activity -a claude-code`. Or copy the skill folder (.github/skills/ai-agent-activity in SCStelz/security-investigator) into .claude/skills/ai-agent-activity in your project. Claude Code loads it when a task matches its description.
Run `npx skills add SCStelz/security-investigator --skill ai-agent-activity -a codex`. Or copy the skill folder (.github/skills/ai-agent-activity in SCStelz/security-investigator) into .agents/skills/ai-agent-activity in your project. Codex loads it when a task matches its description.
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add SCStelz/security-investigator --skill ai-agent-activity -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/ai-agent-activity, .gemini/skills/ai-agent-activity, .github/skills/ai-agent-activity and .opencode/skills/ai-agent-activity in your project.
SKILL.md names no scripts, command-line tools or credentials: AI Agent Activity is instructions for the agent only.
SKILL.md names 2 domains. In commands or code: security.microsoft.com; the agent is likely to contact it when it follows the instructions. As links in the text: learn.microsoft.com. This is read from the text; nothing was executed.
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.
AI Agent Activity is published under the MIT licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.
About 17k tokens (SKILL.md is roughly 67k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.
Skills that share tags, products or a category with AI Agent Activity: Security Guide (jnMetaCode/shellward, 140 stars), China AI Compliance Audit (jnMetaCode/shellward, 140 stars), Prompt Injection Defense (sickn33/agentic-awesome-skills, 47k stars) and Moai Ref LLM Security (modu-ai/moai-adk, 1.2k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
SCStelz (a GitHub user) maintains it in SCStelz/security-investigator, which has 249 GitHub stars. The repository holds 22 skills in this directory. The repository was last updated on October 8, 2026.
Source: SCStelz/security-investigator on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.