Agent skill

Polymarket Brier

by runesleo in runesleo/polymarket-toolkit

Brier Score calculator for Polymarket addresses — measures prediction accuracy independent of PnL.

MITAuto-check passedBusiness, Finance & HR

Install Polymarket Brier

skills CLI
$ npx skills add runesleo/polymarket-toolkit --skill polymarket-brier -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install runesleo/polymarket-toolkit polymarket-brier --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/runesleo/polymarket-toolkit.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/polymarket-brier .claude/skills/polymarket-brier && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
polymarket-brier
GitHub stars
194
Token cost
~1.3k tokens
SKILL.md length
517 words
Files
1
Skills in repo
2
Repo updated
First seen
Licence
MIT

At a glance

Brier Score calculator for Polymarket addresses — measures prediction accuracy independent of PnL.

  • Works in 5 steps: Fetch Settled Positions → Determine Actual Outcome → Calculate Brier Score → …
  • Business, Finance & HR work in your project
  • SKILL.md covers Why Brier Score?, Trigger, Background and Input, plus 2 more sections
  • Calls curl; reaches data-api.polymarket.com

What it does

Polymarket Brier is an agent skill from runesleo/polymarket-toolkit. Brier Score calculator for Polymarket addresses — measures prediction accuracy independent of PnL. Separates skilled predictors from market makers and arbitrageurs.

Its SKILL.md is about 1.3k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.

It sits in Business, Finance & HR. It works with Polymarket and TypeScript. The repository describes itself as: CLI + AI Skills workflow for profiling Polymarket traders from an address. The licence is MIT.

When your agent uses it

  • Business, Finance & HR work in your project

Example prompts

  • “/polymarket-brier”

Requirements

  • Pre-approved tools (allowed-tools): Bash(curl:*), Read, Write, Edit

Workflow steps

5 steps, taken from the step headings in SKILL.md.

  1. Fetch Settled Positions
  2. Determine Actual Outcome
  3. Calculate Brier Score
  4. Calculate Calibration Breakdown (Optional, if ≥20 settled positions)
  5. Output

What it can do on your machine

Read from SKILL.md and the folder at commit 4a66e7d. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves these tools, so the agent can use them without asking each time:

    • Bash(curl:*)
    • Read
    • Write
    • Edit

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Shell commands in SKILL.md call:

    • curl

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    Hosts in commands or code, which the agent is likely to contact:

    • data-api.polymarket.com

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Polymarket Brier loads about 1.3k tokens when it runs. Until then it costs about 45 tokens; SKILL.md has 517 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~45
When it runs · the whole SKILL.md, loaded when a task matches
~1.3k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from runesleo/polymarket-toolkit at commit 4a66e7d, republished under its MIT licence (© runesleo). 517 words, ~1,349 tokens.

Download SKILL.mdSave it as .claude/skills/polymarket-brier/SKILL.md (or your agent's skills folder).
name
polymarket-brier
description
Brier Score calculator for Polymarket addresses — measures prediction accuracy independent of PnL. Separates skilled predictors from market makers and arbitrageurs.
allowed-tools
Bash(curl:*), Read, Write, Edit
metadata.version
0.2.0

Polymarket Brier Score

Calculate prediction accuracy for any Polymarket address. Brier Score measures how close a trader's positions were to the actual outcomes — independent of PnL, position size, or trading strategy.

Why Brier Score?

PnL tells you who made money. Brier Score tells you who predicted correctly.

  • High PnL + Poor Brier → Market maker or arbitrageur (earns spread, not predictions)
  • Low PnL + Good Brier → Accurate predictor with small positions or conservative sizing
  • High PnL + Good Brier → Skilled predictor who also sizes well — the real signal

Trigger

User asks about prediction accuracy, Brier score, prediction quality, forecasting skill, or "how accurate is this trader".

Background

The Brier Score is a standard metric in forecasting (used by Metaculus, Good Judgment Project, etc.).

Formula: BS = (1/N) × Σ(forecast_probability - actual_outcome)²

  • 0.0 = perfect prediction (always right with 100% confidence)
  • 0.25 = no skill (equivalent to random 50/50 guessing)
  • 0.5+ = worse than random

Lower is better.

Input

Same as polymarket-profile — accepts 0x address, username, or profile URL.

Execution

Step 1: Fetch Settled Positions
bash
curl -s "https://data-api.polymarket.com/positions?user={ADDRESS}&sizeThreshold=0&limit=100&offset=0"

Paginate until all positions are fetched (same as polymarket-profile).

Filter to settled positions only: redeemable == true.

For each settled position, extract:

  • outcome: "Yes" or "No" — the side the trader held
  • avgPrice: the trader's average entry price = their implied probability forecast
  • currentValue > 0: whether the market resolved in their favor (won) or not (lost)
Step 2: Determine Actual Outcome

For each settled position:

  • If currentValue > 0 (won): the outcome the trader bet on did happen → actual = 1
  • If currentValue == 0 (lost): the outcome the trader bet on did not happen → actual = 0
Step 3: Calculate Brier Score

For each settled position:

forecast = avgPrice  (what they paid = their implied probability)
squared_error = (forecast - actual)²

Then:

brier_score = sum(squared_error) / count(settled_positions)
Step 4: Calculate Calibration Breakdown (Optional, if ≥20 settled positions)

Group positions into probability buckets by avgPrice:

BucketRangeMeaning
High confidence0.80-0.99"Very likely to happen"
Moderate0.60-0.79"Probably happens"
Coin flip0.40-0.59"Could go either way"
Contrarian0.01-0.39"Betting against the crowd"

For each bucket:

  • Count positions
  • Average forecast probability
  • Actual win rate (% that resolved in their favor)
  • A well-calibrated trader's win rate should be close to their average forecast
Show full SKILL.md (180 more words)Show less
Step 5: Output
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  Brier Score: {ADDRESS_SHORT}
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

📊 Prediction Accuracy
  Brier Score:      {score} ({rating})
  Settled Markets:  {count}
  Correct:          {wins}/{total} ({win_rate}%)

📐 Calibration
  | Confidence | Positions | Avg Forecast | Actual Win% | Gap |
  |------------|-----------|-------------|-------------|-----|
  | High       | 12        | 87%         | 83%         | -4% |
  | Moderate   | 8         | 68%         | 62%         | -6% |
  | Coin flip  | 5         | 52%         | 60%         | +8% |
  | Contrarian | 3         | 28%         | 33%         | +5% |

💡 Interpretation
  {1-2 sentence plain language summary}

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  Generated: {timestamp} | Data: Polymarket Public APIs
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Rating Scale
Brier ScoreRatingMeaning
0.00-0.10ExcellentTop-tier forecaster
0.10-0.15GoodConsistently better than chance
0.15-0.20AverageSome skill, room to improve
0.20-0.25PoorBarely better than guessing
0.25+No skillWorse than a coin flip

Notes

  • Brier Score requires settled positions. Accounts with mostly open positions will have a small sample size — flag this: "Based on N settled markets. Score reliability increases with more data."
  • If fewer than 5 settled positions, output: "Not enough settled data for a meaningful Brier Score."
  • avgPrice is used as the probability forecast. This is a simplification — traders who DCA (buy at multiple prices) will have a blended avgPrice. Acceptable for v0.2.
  • SPLIT positions should be excluded from Brier calculation. SPLIT is a market-neutral operation (buy both sides), not a directional prediction. Identify via polymarket-profile's activity data if available, otherwise include all positions with a note.
  • This score is independent of position size. A $10 bet and a $10,000 bet on the same market at the same price contribute equally. This is intentional — we're measuring prediction skill, not bankroll management.

© runesleo, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in skills/polymarket-brier of runesleo/polymarket-toolkit.

Open the folder on GitHubat commit 4a66e7d

Compare with similar skills

Polymarket Brier next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Polymarket Brier compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Polymarket Brier this skillrunesleo/polymarket-toolkit194—~1.3kAutomated safety check: PassMIT
Polymarket Tennislivetennisapi/livetennisapi-mcp152—~3kAutomated safety check: PassMIT
Web3 PolymarketPolymarket/agent-skills1921 repos~2kAutomated safety check: PassNone
Digital Oraclekomako-workshop/digital-oracle878—~5.9kAutomated safety check: PassMIT
Polyclawchainstacklabs/polyclaw3591 repos~2kAutomated safety check: PassApache-2.0
Polymarket TradingBlockRunAI/ClawRouter6.6k—~1.4kAutomated safety check: PassMIT

Similar skills

  • Polymarket Tennis

    livetennisapi/livetennisapi-mcp

    Build observe-only Polymarket and Kalshi tennis market tooling on the polymarket-tennis Python package (MIT) plus the Live Tennis API free tier.

    152 GitHub stars~3k tokensUpdated 4 days ago
    Business, Finance & HRAuto-check passed
  • Web3 Polymarket

    Polymarket/agent-skills

    Polymarket integration for prediction market trading on Polygon.

    192 GitHub starsUsed in 1 repo~2k tokens
    Backend & APIsAuto-check passed
  • Digital Oracle

    komako-workshop/digital-oracle

    Answer prediction questions using market trading data, not opinions.

    878 GitHub stars~5.9k tokensUpdated 2 mo ago
    Business, Finance & HRAuto-check passed
  • Polyclaw

    chainstacklabs/polyclaw

    Trade on Polymarket via split + CLOB execution. An agent skill from chainstacklabs/polyclaw.

    359 GitHub starsUsed in 1 repo~2k tokens
    Business, Finance & HRAuto-check passed
  • Polymarket Trading

    BlockRunAI/ClawRouter

    A skill your agent uses when the user wants to actually PLACE, manage, or redeem bets on Polymarket (not just read odds — that's the blockrunpredexon data tools).

    6.6k GitHub stars~1.4k tokensUpdated 5 days ago
    Business, Finance & HRAuto-check passed
  • Dr Manhattan

    guzus/dr-manhattan

    Trade prediction markets (Polymarket, Kalshi, Opinion, Limitless, Predict.fun) using a unified CCXT-style API.

    204 GitHub stars~2k tokensUpdated 2 mo ago
    Business, Finance & HRAuto-check passed

More from runesleo/polymarket-toolkit

  • Polymarket Profile

    runesleo/polymarket-toolkit

    Polymarket address profiler — input any 0x address, get a complete trading profile with PnL, win rate, positions, category breakdown, and top trades.

    194 GitHub stars~3.7k tokensUpdated 1 mo ago
    Auto-check passed

Questions about Polymarket Brier

What does Polymarket Brier do?

Brier Score calculator for Polymarket addresses — measures prediction accuracy independent of PnL. Polymarket Brier is an agent skill from runesleo/polymarket-toolkit. Brier Score calculator for Polymarket addresses — measures prediction accuracy independent of PnL.

When should I use Polymarket Brier?

Polymarket Brier fits situations like: business, Finance & HR work in your project.

How do I install Polymarket Brier in Claude Code?

Run `npx skills add runesleo/polymarket-toolkit --skill polymarket-brier -a claude-code`. Or copy the skill folder (skills/polymarket-brier in runesleo/polymarket-toolkit) into .claude/skills/polymarket-brier in your project. Claude Code loads it when a task matches its description.

How do I install Polymarket Brier in Codex?

Run `npx skills add runesleo/polymarket-toolkit --skill polymarket-brier -a codex`. Or copy the skill folder (skills/polymarket-brier in runesleo/polymarket-toolkit) into .agents/skills/polymarket-brier in your project. Codex loads it when a task matches its description.

Can I use Polymarket Brier in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add runesleo/polymarket-toolkit --skill polymarket-brier -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/polymarket-brier, .gemini/skills/polymarket-brier, .github/skills/polymarket-brier and .opencode/skills/polymarket-brier in your project.

What does Polymarket Brier need to run?

Going by SKILL.md and its folder, Polymarket Brier needs the command-line tools its instructions call (curl). Its frontmatter pre-approves these tools: Bash(curl:*), Read, Write, Edit.

Does Polymarket Brier access the network?

SKILL.md names 1 domain. In commands or code: data-api.polymarket.com; the agent is likely to contact it when it follows the instructions. This is read from the text; nothing was executed.

Is Polymarket Brier safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Polymarket Brier use?

Polymarket Brier is published under the MIT licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Polymarket Brier use?

About 1.3k tokens (SKILL.md is roughly 5.4k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Polymarket Brier?

Skills that share tags, products or a category with Polymarket Brier: Polymarket Tennis (livetennisapi/livetennisapi-mcp, 152 stars), Web3 Polymarket (Polymarket/agent-skills, 192 stars), Digital Oracle (komako-workshop/digital-oracle, 878 stars) and Polyclaw (chainstacklabs/polyclaw, 359 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Polymarket Brier?

runesleo (a GitHub user) maintains it in runesleo/polymarket-toolkit, which has 194 GitHub stars. The repository holds 2 skills in this directory. The repository was last updated on September 9, 2026.

Source: runesleo/polymarket-toolkit on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.