Agent skill

Agentmeasure

by roy-tong in roy-tong/AgentMeasure

Check whether agent telemetry preserves measurement semantics.

MITAuto-check passedDevOps & Cloud

Install Agentmeasure

skills CLI
$ npx skills add roy-tong/AgentMeasure --skill agentmeasure -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install roy-tong/AgentMeasure agentmeasure --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/roy-tong/AgentMeasure.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/agentmeasure .claude/skills/agentmeasure && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
agentmeasure
GitHub stars
218
Token cost
~753 tokens
SKILL.md length
226 words
Files
1
Skills in repo
1
Repo updated
First seen
Licence
MIT

At a glance

Check whether agent telemetry preserves measurement semantics.

  • A user asks whether retries inflate counted usage
  • SKILL.md covers First success (offline,…, What to check, and where and Boundaries
  • Calls python3 and git; reaches raw.githubusercontent.com
  • Whether an operation differs from an attempt

What it does

Agentmeasure is an agent skill from roy-tong/AgentMeasure. Check whether agent telemetry preserves measurement semantics. Use when a user asks whether retries inflate counted usage, whether an operation differs from an attempt, whether token metrics double count a subset (reasoning inside output), whether cache hits or cached replays distort counts, or wants conformance tests for agent metrics and observability claims. 中文触发:遥测语义核查、重试计数、用量口径、token 重复计数、Agent 用量指标一致性检验。

Its SKILL.md is about 750 tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.

It sits in DevOps & Cloud, covering Observability. It works with Model Context Protocol and OpenTelemetry. The repository describes itself as: Independent verification of AI support bills: applicable terms, invoice and business records, with reviewable findings. Open-source conformance for agent telemetry — PASS / FAIL… The licence is MIT.

When your agent uses it

  • A user asks whether retries inflate counted usage
  • Whether an operation differs from an attempt
  • Whether token metrics double count a subset (reasoning inside output)
  • Whether cache hits

Example prompts

  • “/agentmeasure”

Requirements

  • Python 3

What it can do on your machine

Read from SKILL.md and the folder at commit 67c276b. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Shell commands in SKILL.md call:

    • python3
    • git

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    Hosts in commands or code, which the agent is likely to contact:

    • raw.githubusercontent.com

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Agentmeasure loads about 753 tokens when it runs. Until then it costs about 107 tokens; SKILL.md has 226 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~107
When it runs · the whole SKILL.md, loaded when a task matches
~753

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from roy-tong/AgentMeasure at commit 67c276b, republished under its MIT licence (© roy-tong). 226 words, ~753 tokens.

Download SKILL.mdSave it as .claude/skills/agentmeasure/SKILL.md (or your agent's skills folder).
name
agentmeasure
description
Check whether agent telemetry preserves measurement semantics. Use when a user asks whether retries inflate counted usage, whether an operation differs from an attempt, whether token metrics double count a subset (reasoning inside output), whether cache hits or cached replays distort counts, or wants conformance tests for agent metrics and observability claims. 中文触发:遥测语义核查、重试计数、用量口径、token 重复计数、Agent 用量指标一致性检验。

AgentMeasure — Measurement Conformance

This Skill routes measurement-semantics questions to AgentMeasure's local runners. It does not move telemetry anywhere.

First success (offline, deterministic)

bash
git clone https://github.com/roy-tong/AgentMeasure && cd AgentMeasure
python3 conformance/pack/agentmeasure selftest          # Pack fixtures, PASS / FAIL / UNPROVABLE
python3 conformance/pack/agentmeasure conformance \
  --fixture conformance/vectors/external/urusilla-001/agentmeasure_urusilla_fixture_001.events.jsonl
./examples/demo-e2e.sh                                  # 42 calls -> 84 canonical observations, deterministic
python3 conformance/runners/run_metrics.py              # 21/21 vectors PASS
python3 verify_vectors.py                               # receipts / correlation / operation grouping

These commands run locally. The Pack reads the caller-supplied FMT-002 fixture; selftest is deliberately separate and only checks repository fixtures. No network, cloud, or credentials are required.

What to check, and where

QuestionRunner / artifact
Do retries inflate counted usage? (one operation vs N attempts)conformance/vectors/metric-execution-grain.json
Does a selection rate claim hold at its grain?conformance/vectors/metric-selection-rate.json
Do declared operation summaries reconcile against attempt rows?conformance/vectors/external/urusilla-001/
Does an operation-grain metric aggregate at the wrong grain?conformance/vectors/external/urusilla-002/
Can this trace safely support an operation count at all?conformance/evidence/langfuse-demo-traces/ (the 0%-coverage case)

Verdict vocabulary: PASS / FAIL / UNPROVABLE. Ambiguity is retained in the reason for an UNPROVABLE finding; it is not a fourth public verdict. Report UNPROVABLE as a finding, never as zero.

Boundaries

  • Local by default. Nothing is uploaded; a sanitized sample is shared only with explicit authorization.
  • A short sample check is not a full audit and claims no causal effect.
  • The Pack CLI and root GitHub Action are shipped on main. Until a release tag explicitly includes the Pack, pin the Action to the verified immutable commit 2cf476d6f7d0fc45401db5a822e1f12de009ac74 rather than a moving branch.
  • Provider-side usage questions (who called, how often, production traffic) start with the zero-install check described in the repository README.

Catalog entry: https://raw.githubusercontent.com/roy-tong/roy-tong/main/agent-tools.json

© roy-tong, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in skills/agentmeasure of roy-tong/AgentMeasure.

Open the folder on GitHubat commit 67c276b

Compare with similar skills

Agentmeasure next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Agentmeasure compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Agentmeasure this skillroy-tong/AgentMeasure218—~753Automated safety check: PassMIT
UModel Root Cause Analysisalibaba/UnifiedModel415—~1.9kAutomated safety check: PassCustom licence
Exploring Apm TracesPostHog/posthog40k—~3.5kAutomated safety check: PassCustom licence
Archestra Dev Observabilityarchestra-ai/archestra4.4k—~1.2kAutomated safety check: PassCustom licence
Frontmcp Observabilityagentfront/frontmcp146—~4.6kAutomated safety check: PassApache-2.0
API Telemetrycyanheads/pubmed-mcp-server156—~8.7kAutomated safety check: PassApache-2.0

Similar skills

  • UModel Root Cause Analysis

    alibaba/UnifiedModel

    Investigates a service incident to its root cause by querying a UModel object graph alongside metrics, logs, topology and recent deployments.

    415 GitHub stars~1.9k tokensUpdated 15 days ago
    DevOps & CloudAuto-check passed
  • Exploring Apm Traces

    PostHog/posthog

    Official

    Investigates distributed application performance using PostHog APM (OpenTelemetry span) data via MCP.

    40k GitHub stars~3.5k tokensUpdated today
    DevOps & CloudAuto-check passed
  • Archestra Dev Observability

    archestra-ai/archestra

    A skill your agent uses when changing Archestra tracing, metrics, OpenTelemetry, Tempo, Grafana, Prometheus, LLM/MCP spans, observability labels, or local observability setup.

    4.4k GitHub stars~1.2k tokensUpdated today
    DevOps & CloudAuto-check passed
  • Frontmcp Observability

    agentfront/frontmcp

    A skill your agent uses when adding tracing, structured logging, metrics, or monitoring to a FrontMCP server.

    146 GitHub stars~4.6k tokensUpdated 2 days ago
    DevOps & CloudAuto-check passed
  • API Telemetry

    cyanheads/pubmed-mcp-server

    Catalog of OpenTelemetry instrumentation built into framework @cyanheads/mcp-ts-core — spans, metrics, completion logs, env config, runtime caveats, custom instrumentation patterns, and cardinality…

    156 GitHub stars~8.7k tokensUpdated 5 days ago
    DevOps & CloudAuto-check passed
  • Genai Observability

    VectorSpaceLab/AREX-Skill

    A skill your agent uses for MLflow GenAI observability work: tracing, trace search/export, OpenTelemetry, prompts, GenAI datasets/evaluation, scorers/judges, review queues, assessments, and provider…

    330 GitHub stars~726 tokensUpdated 1 mo ago
    DevOps & CloudAuto-check passed

Categories

Questions about Agentmeasure

What does Agentmeasure do?

Check whether agent telemetry preserves measurement semantics. Agentmeasure is an agent skill from roy-tong/AgentMeasure. Check whether agent telemetry preserves measurement semantics.

When should I use Agentmeasure?

Agentmeasure fits situations like: A user asks whether retries inflate counted usage; whether an operation differs from an attempt; whether token metrics double count a subset (reasoning inside output); whether cache hits.

How do I install Agentmeasure in Claude Code?

Run `npx skills add roy-tong/AgentMeasure --skill agentmeasure -a claude-code`. Or copy the skill folder (skills/agentmeasure in roy-tong/AgentMeasure) into .claude/skills/agentmeasure in your project. Claude Code loads it when a task matches its description.

How do I install Agentmeasure in Codex?

Run `npx skills add roy-tong/AgentMeasure --skill agentmeasure -a codex`. Or copy the skill folder (skills/agentmeasure in roy-tong/AgentMeasure) into .agents/skills/agentmeasure in your project. Codex loads it when a task matches its description.

Can I use Agentmeasure in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add roy-tong/AgentMeasure --skill agentmeasure -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/agentmeasure, .gemini/skills/agentmeasure, .github/skills/agentmeasure and .opencode/skills/agentmeasure in your project.

What does Agentmeasure need to run?

Going by SKILL.md and its folder, Agentmeasure needs the command-line tools its instructions call (python3 and git). Our summary lists: Python 3.

Does Agentmeasure access the network?

SKILL.md names 1 domain. In commands or code: raw.githubusercontent.com; the agent is likely to contact it when it follows the instructions. This is read from the text; nothing was executed.

Is Agentmeasure safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Agentmeasure use?

Agentmeasure is published under the MIT licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Agentmeasure use?

About 753 tokens (SKILL.md is roughly 3k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Agentmeasure?

Skills that share tags, products or a category with Agentmeasure: UModel Root Cause Analysis (alibaba/UnifiedModel, 415 stars), Exploring Apm Traces (PostHog/posthog, 40k stars), Archestra Dev Observability (archestra-ai/archestra, 4.4k stars) and Frontmcp Observability (agentfront/frontmcp, 146 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Agentmeasure?

roy-tong (a GitHub user) maintains it in roy-tong/AgentMeasure, which has 218 GitHub stars. The repository was last updated on October 8, 2026.

Source: roy-tong/AgentMeasure on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.