Agent skill

Confidence Calibration Check

by GarethManning in GarethManning/education-agent-skills

Capture confidence ratings before and after a learning attempt to identify overconfidence and underconfidence patterns.

Custom licenceAuto-check passedBusiness, Finance & HR

Install Confidence Calibration Check

skills CLI
$ npx skills add GarethManning/education-agent-skills --skill confidence-calibration-check -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install GarethManning/education-agent-skills confidence-calibration-check --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/GarethManning/education-agent-skills.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/student-learning/confidence-calibration-check .claude/skills/confidence-calibration-check && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
confidence-calibration-check
GitHub stars
840
Token cost
~3.8k tokens
SKILL.md length
1,006 words
Files
1
Skills in repo
160
Repo updated
First seen
Licence
Custom licence

At a glance

Capture confidence ratings before and after a learning attempt to identify overconfidence and underconfidence patterns.

  • Works in 4 steps: The 0–100 confidence scale is intuitive… → The calibration pattern depends on the… → Frequent calibration checks can produce… → …
  • A student wants to understand how well they actually know something versus how well they think they know it
  • SKILL.md covers What This Skill Does, Evidence Foundation, System Prompt and Example Transcript, plus 1 more section
  • Instructions only: no scripts, shell commands, URLs or credentials in SKILL.md

What it does

Confidence Calibration Check is an agent skill from GarethManning/education-agent-skills. Capture confidence ratings before and after a learning attempt to identify overconfidence and underconfidence patterns. Use when a student wants to understand how well they actually know something versus how well they think they know it.

Its SKILL.md is about 3.8k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.

It sits in Business, Finance & HR, covering Performance reviews. The repository describes itself as: 165 evidence-grounded AI skills for teachers, school leaders and EdTech builders—pedagogy, learning science, curriculum, assessment and regeneration. Claude, Codex and Hermes.

When your agent uses it

  • A student wants to understand how well they actually know something versus how well they think they know it
  • Tasks that involve Performance reviews

Example prompts

  • “/confidence-calibration-check”

Workflow steps

4 steps, taken from the first numbered list in SKILL.md.

  1. The 0–100 confidence scale is intuitive but not psychometrically calibrated. Different learners interpret the scale differently. A 70 from…
  2. The calibration pattern depends on the quality of the performance evaluation. If the AI misjudges the accuracy of the learner's attempt…
  3. Frequent calibration checks can produce strategic responding if learners game the rating. A learner who understands the pattern may…
  4. Calibration data across sessions requires session history to be useful. A single calibration check gives a snapshot. The pattern…

What it can do on your machine

Read from SKILL.md and the folder at commit 6bbbce4. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    No scripts in the folder and no shell commands in SKILL.md.

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    No URLs in SKILL.md.

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Confidence Calibration Check loads about 3.8k tokens when it runs. Until then it costs about 67 tokens; SKILL.md has 1,006 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~67
When it runs · the whole SKILL.md, loaded when a task matches
~3.8k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

Its licence (Custom licence) doesn't allow us to republish the file, so here is its outline and opening line. It has 1,006 words (~3,849 tokens).

“Captures a confidence rating (0–100) before a knowledge attempt and again after receiving feedback, then compares the two. The AI identifies and names the pattern: overconfidence (high confidence + poor performance) and underconfidence (low confidence + good performance) are both…”

— opening of SKILL.md by GarethManning, Custom licence
name
confidence-calibration-check
disable-model-invocation
false
user-invocable
true
effort
light
skill_id
student-learning/confidence-calibration-check
skill_name
Confidence Calibration Check
domain
student-learning
domain_number
20
version
1.0
audience
student
evidence_strength
moderate

Read the full SKILL.md on GitHub

Files

Just SKILL.md in skills/student-learning/confidence-calibration-check of GarethManning/education-agent-skills.

Open the folder on GitHubat commit 6bbbce4

Compare with similar skills

Confidence Calibration Check next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Confidence Calibration Check compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Confidence Calibration Check this skillGarethManning/education-agent-skills840—~3.8kAutomated safety check: PassCustom licence
Hwb Review Paperxieyouxixi/hwb-math-modeling-skills197—~3.2kAutomated safety check: PassNone
Wp Performance Reviewelvismdev/claude-wordpress-skills2351 repos~4.5kAutomated safety check: PassMIT
Coachnagisanzenin/engram1.4k—~10kAutomated safety check: PassMIT
Align Humanagentscope-ai/OpenJudge871—~3.1kAutomated safety check: PassApache-2.0
Run Mv Hoi Reconstructionnvidia-isaac/video_to_data861—~1.5kAutomated safety check: PassCustom licence

Similar skills

  • Hwb Review Paper

    xieyouxixi/hwb-math-modeling-skills

    Review mathematical modeling competition papers from a complete problem and paper, with an optional strict hwb-paper-format-checker audit, itemized problem-specific scoring, a bottom-up low-score…

    197 GitHub stars~3.2k tokensUpdated 24 days ago
    Business, Finance & HRAuto-check passed
  • Wp Performance Review

    elvismdev/claude-wordpress-skills

    WordPress performance code review and optimization analysis.

    235 GitHub starsUsed in 1 repo~4.5k tokens
    Business, Finance & HRAuto-check passed
  • Coach

    nagisanzenin/engram

    Learning telemetry, strategy, and schedule — retention stats, calibration, grader audit, n-of-1 experiments, HTML dashboard.

    1.4k GitHub stars~10k tokensUpdated 1 mo ago
    EducationAuto-check passed
  • Align Human

    agentscope-ai/OpenJudge

    A skill your agent uses when the user has a judge/grader and human-labeled data, and wants to measure how well the judge agrees with humans, detect systematic biases, determine whether automatic…

    871 GitHub stars~3.1k tokensUpdated 1 mo ago
    Business, Finance & HRAuto-check passed
  • Run Mv Hoi Reconstruction

    nvidia-isaac/video_to_data

    Run and validate the repository-local multi-view camera calibration and human-object reconstruction pipelines.

    861 GitHub stars~1.5k tokensUpdated 2 days ago
    Business, Finance & HRAuto-check passed
  • Company Analysis

    zhu1090093659/dsh-trading

    A skill your agent uses when the user wants to analyze a listed company, stock, business, or investment target; challenge or revise an existing company report; compare A/H or primary-listing/ADR…

    238 GitHub stars~4.2k tokensUpdated today
    Business, Finance & HRAuto-check passed

More from GarethManning/education-agent-skills

All 160 skills in this repo
  • Academic Language Sentence Frame Generator

    GarethManning/education-agent-skills

    Generate tiered sentence frames for academic tasks that scaffold language production across proficiency levels.

    840 GitHub stars~5.1k tokensUpdated 1 mo ago
    Auto-check passed
  • Adaptive Hint Sequence Designer

    GarethManning/education-agent-skills

    Generate a cascading hint sequence for a problem type, revealing progressively without giving answers.

    840 GitHub stars~5.4k tokensUpdated 1 mo ago
    Auto-check passed
  • Agency Circles For Systems Action

    GarethManning/education-agent-skills

    Map control, influence, and concern after systems analysis. An agent skill from GarethManning/education-agent-skills.

    840 GitHub stars~2.3k tokensUpdated 1 mo ago
    Auto-check passed
  • Agency Scaffold Generator

    GarethManning/education-agent-skills

    Generate scaffolds that gradually increase student choice, voice, and ownership within a learning task.

    840 GitHub stars~5.6k tokensUpdated 1 mo ago
    Auto-check passed
  • AI Expertise Interrogation Designer

    GarethManning/education-agent-skills

    Design a Funhouse Mirror activity where students use their own domain expertise to detect AI distortions, omissions, and overconfidence.

    840 GitHub stars~5.8k tokensUpdated 1 mo ago
    Auto-check passed
  • AI Feedback Design Principles

    GarethManning/education-agent-skills

    Audit and redesign AI-generated feedback for pedagogical quality, timing, and learning impact.

    840 GitHub stars~5.8k tokensUpdated 1 mo ago
    Auto-check passed

Questions about Confidence Calibration Check

What does Confidence Calibration Check do?

Capture confidence ratings before and after a learning attempt to identify overconfidence and underconfidence patterns. Confidence Calibration Check is an agent skill from GarethManning/education-agent-skills. Capture confidence ratings before and after a learning attempt to identify overconfidence and underconfidence patterns.

When should I use Confidence Calibration Check?

Confidence Calibration Check fits situations like: A student wants to understand how well they actually know something versus how well they think they know it; tasks that involve Performance reviews.

How do I install Confidence Calibration Check in Claude Code?

Run `npx skills add GarethManning/education-agent-skills --skill confidence-calibration-check -a claude-code`. Or copy the skill folder (skills/student-learning/confidence-calibration-check in GarethManning/education-agent-skills) into .claude/skills/confidence-calibration-check in your project. Claude Code loads it when a task matches its description.

How do I install Confidence Calibration Check in Codex?

Run `npx skills add GarethManning/education-agent-skills --skill confidence-calibration-check -a codex`. Or copy the skill folder (skills/student-learning/confidence-calibration-check in GarethManning/education-agent-skills) into .agents/skills/confidence-calibration-check in your project. Codex loads it when a task matches its description.

Can I use Confidence Calibration Check in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add GarethManning/education-agent-skills --skill confidence-calibration-check -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/confidence-calibration-check, .gemini/skills/confidence-calibration-check, .github/skills/confidence-calibration-check and .opencode/skills/confidence-calibration-check in your project.

What does Confidence Calibration Check need to run?

SKILL.md names no scripts, command-line tools or credentials: Confidence Calibration Check is instructions for the agent only.

Does Confidence Calibration Check access the network?

SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.

Is Confidence Calibration Check safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Confidence Calibration Check use?

Confidence Calibration Check has a licence file (the repository's licence) that doesn't match a standard licence. Read it on GitHub before reusing the skill.

How many tokens does Confidence Calibration Check use?

About 3.8k tokens (SKILL.md is roughly 15k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Confidence Calibration Check?

Skills that share tags, products or a category with Confidence Calibration Check: Hwb Review Paper (xieyouxixi/hwb-math-modeling-skills, 197 stars), Wp Performance Review (elvismdev/claude-wordpress-skills, 235 stars), Coach (nagisanzenin/engram, 1.4k stars) and Align Human (agentscope-ai/OpenJudge, 871 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Confidence Calibration Check?

GarethManning (a GitHub user) maintains it in GarethManning/education-agent-skills, which has 840 GitHub stars. The repository holds 160 skills in this directory. The repository was last updated on August 28, 2026.

Source: GarethManning/education-agent-skills on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.