Tests Query Corpus
netdata/netdata
Run, extend or review the Netdata query contract corpus (tests/query-corpus), its fixtures, independent oracles, byte pins and harness.
Ranked content search over a text corpus you point it at, using BM25 (via xhluca/bm25s).
MITAuto-check passed
Install Bm25
$ npx skills add oaustegard/claude-skills --skill bm25 -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install oaustegard/claude-skills bm25 --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/oaustegard/claude-skills.git skills-src && mkdir -p .claude/skills && cp -r skills-src/bm25 .claude/skills/bm25 && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "bm25" agent skill from https://github.com/oaustegard/claude-skills/tree/main/bm25 into .claude/skills/bm25/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "bm25", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/oaustegard/claude-skills/tree/main/bm25Type this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add oaustegard/claude-skills --skill bm25 -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install oaustegard/claude-skills bm25 --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/oaustegard/claude-skills.git skills-src && mkdir -p .agents/skills && cp -r skills-src/bm25 .agents/skills/bm25 && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "bm25" agent skill from https://github.com/oaustegard/claude-skills/tree/main/bm25 into .agents/skills/bm25/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "bm25", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add oaustegard/claude-skills --skill bm25 -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install oaustegard/claude-skills bm25 --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/oaustegard/claude-skills.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/bm25 .cursor/skills/bm25 && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "bm25" agent skill from https://github.com/oaustegard/claude-skills/tree/main/bm25 into .cursor/skills/bm25/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "bm25", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/oaustegard/claude-skills.git --path bm25--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add oaustegard/claude-skills --skill bm25 -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install oaustegard/claude-skills bm25 --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/oaustegard/claude-skills.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/bm25 .gemini/skills/bm25 && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "bm25" agent skill from https://github.com/oaustegard/claude-skills/tree/main/bm25 into .gemini/skills/bm25/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "bm25", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install oaustegard/claude-skills bm25Installs for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add oaustegard/claude-skills --skill bm25 -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/oaustegard/claude-skills.git skills-src && mkdir -p .github/skills && cp -r skills-src/bm25 .github/skills/bm25 && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "bm25" agent skill from https://github.com/oaustegard/claude-skills/tree/main/bm25 into .github/skills/bm25/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "bm25", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add oaustegard/claude-skills --skill bm25 -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install oaustegard/claude-skills bm25 --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/oaustegard/claude-skills.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/bm25 .opencode/skills/bm25 && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "bm25" agent skill from https://github.com/oaustegard/claude-skills/tree/main/bm25 into .opencode/skills/bm25/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "bm25", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
Facts
- Skill name
bm25- GitHub stars
- 150
- Token cost
- ~1.7k tokens
- SKILL.md length
- 534 words
- Files
- 4 (incl. scripts)
- Skills in repo
- 67
- Repo updated
- First seen
- Licence
- MIT
At a glance
Ranked content search over a text corpus you point it at, using BM25 (via xhluca/bm25s).
- Rank these documents
- SKILL.md covers Setup, Usage, Corpus types and Options, plus 4 more sections
- Runs Python scripts from its folder; calls python3 and uv; needs GH_TOKEN
- Search this corpus
What it does
Bm25 is an agent skill from oaustegard/claude-skills. Ranked content search over a text corpus you point it at, using BM25 (via xhluca/bm25s). Corpus-agnostic: cloned repos, project knowledge stores, uploaded files and archives, any local directory. In-memory BM25 index per invocation, with a session-local disk cache for repeat runs against the same corpus. Use for "rank these documents", "search this corpus", "which files are most about X", "find content about Y", or any multi-word concept query against a known body of text where grep would return everything or…
Its SKILL.md is about 1.7k tokens, which your agent loads only when the skill is triggered. The skill folder holds 4 other files, including scripts (for example `CHANGELOG.md`, `README.md` and `scripts/bm25.py`).
The repository describes itself as: My collection of Claude skills. The licence is MIT.
When your agent uses it
- Rank these documents
- Search this corpus
- Which files are most about X
- Find content about Y
Example prompts
- “rank these documents”
- “search this corpus”
- “which files are most about X”
- “/bm25”
Requirements
- Python 3
What it can do on your machine
Read from SKILL.md and the folder at commit 90b0f1b. It shows what the files ask for, not the result of running them.
Tool permissions
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Runs code
Ships 1 file in scripts/ (Python), which the agent can run.
Shell commands in SKILL.md call:
python3uv
From the folder's file list and the shell code blocks in SKILL.md.
Network
No URLs in SKILL.md. Its commands use uv, which can reach the network depending on how they are called.
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names these keys or tokens, usually read from environment variables:
GH_TOKEN
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Context cost
Bm25 loads about 1.7k tokens when it runs. Until then it costs about 184 tokens; SKILL.md has 534 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
Safety
Auto-check passed
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.
SKILL.md
The full file from oaustegard/claude-skills at commit 90b0f1b, republished under its MIT licence (© oaustegard). 534 words, ~1,747 tokens.
Download SKILL.mdSave it as
.claude/skills/bm25/SKILL.md (or your agent's skills folder). This skill also uses 3 other files; get the full folder from GitHub.- name
- bm25
- description
- Ranked content search over a text corpus you point it at, using BM25 (via xhluca/bm25s). Corpus-agnostic: cloned repos, project knowledge stores, uploaded files and archives, any local directory. In-memory BM25 index per invocation, with a session-local disk cache for repeat runs against the same corpus. Use for "rank these documents", "search this corpus", "which files are most about X", "find content about Y", or any multi-word concept query against a known body of text where grep would return everything or nothing. Needs a corpus on disk. Not for searching stored memories or prior-session decisions (remembering), not for a named symbol or a file's structure (tree-sitting), and not for a literal string you could grep.
- metadata.version
- 0.2.1
bm25
Ranked content search over any text corpus. One CLI, in-memory BM25 index per process, with a session-local disk cache so repeat invocations against the same corpus load in tens of milliseconds instead of rebuilding.
Setup
uv pip install --system --break-system-packages bm25sInstall is sub-second on a warm uv cache. That's the entire dependency.
Usage
BM25=/mnt/skills/user/bm25/scripts/bm25.py
# Local directory
python3 $BM25 ./repo 'csrf middleware'
# Multiple queries against the same in-memory index (build once, query many)
python3 $BM25 ./repo 'csrf middleware' 'session backend' 'queryset filter'
# Cloned GitHub repo via tarball (one HTTP call)
python3 $BM25 'github.com/django/django' 'atomic transaction'
python3 $BM25 'github.com/django/django@stable/5.0.x' 'atomic transaction'
# Project knowledge or uploads
python3 $BM25 project 'RAG scaling laws'
python3 $BM25 uploads 'tax loss harvesting'
# Filters
python3 $BM25 ./repo 'auth flow' --exclude 'tests/*' --exclude '*/tests/*'
python3 $BM25 ./repo 'config' --include '*.py' --include '*.toml'
# Interactive (REPL — single corpus, many queries)
python3 $BM25 ./repo --interactive
# JSON output for piping
python3 $BM25 ./repo 'auth flow' --jsonCorpus types
| Spec | Meaning |
|---|---|
./path or /abs/path | Local directory |
uploads | /mnt/user-data/uploads/ |
project | /mnt/project/ |
github.com/owner/repo[@ref] | Tarball fetch via GitHub API (GH_TOKEN used if set) |
Options
| Option | Default | Description |
|---|---|---|
--top-k N | 10 | Results per query |
--include GLOB | (auto) | Repeatable. If set, only files matching one of these globs are indexed |
--exclude GLOB | Repeatable. Skip files matching these globs | |
--snippet-lines N | 3 | Lines of snippet context per hit (0 = none) |
--max-file-bytes N | 2,000,000 | Skip files larger than this |
--json | Machine-readable output | |
--interactive / -i | REPL mode for ad-hoc querying within one session | |
--stats | Print discover + index timings as JSON | |
--no-cache | Bypass the session-local index cache; build in-memory only |
With no --include, a default set of text/code extensions is indexed (Python,
JS/TS, Go, Rust, Markdown, JSON, YAML, etc.). Standard noise dirs are skipped
unconditionally: .git, node_modules, __pycache__, .venv, dist, etc.
When to use bm25
| Question shape | Tool |
|---|---|
"Find lines matching class.*Error" | grep / ripgrep |
"Show me where parse_input is defined" | tree-sitting (find:/source:) |
| "Which files are about CSRF handling?" | bm25 |
| "Rank these docs by relevance to 'rate limiting strategies'" | bm25 |
| "What's the implementation of the atomic transaction context manager?" | bm25, then tree-sitting source: |
| "Find code by natural-language concept (in a code repo)" | searching-codebases (which has its own TF-IDF mode) |
The boundary with searching-codebases: that skill is code-specific (routes
between regex and TF-IDF, expands via tree-sitting AST). bm25 is the simpler
general-purpose tool — any corpus, no AST awareness, no routing. Prefer
searching-codebases for code; reach for bm25 when the corpus is mixed
(docs + code), non-code (notes, transcripts, PDFs converted to text), or when
you specifically want BM25's length-normalized scoring.
Show full SKILL.md (224 more words)Show less
Design notes
- Session-local disk cache at
/home/claude/.bm25-cache/<key>/. The key is a hash of(resolved_corpus_path, include_globs, exclude_globs, max_file_bytes)— any change invalidates naturally. First invocation builds and saves; subsequent invocations against the same corpus and filters load in tens of milliseconds. The cache lives in/home/claude, which is ephemeral, so it expires at the session boundary — same lifetime as the corpus state itself, no cross-session staleness. ~5–35MB per cached index, depending on corpus size. --no-cachebypasses both load and save — useful only if you've mutated the corpus mid-session (rare) or want to confirm a rebuild matches.- Reuse within a single invocation. The retriever stays in memory
between queries in one process. Passing multiple queries positionally,
or using
--interactive, amortizes any rebuild cost across queries. - No AST awareness. Chunking is per-file. For symbol-level results in
code, combine with
tree-sittingqueries on the same paths. - Tokenizer. Default
bm25s.tokenizewith stopwords disabled — over a small Django sample, AST-derived token streams (identifiers/strings/ comments only) gave near-identical rankings, so we don't bother.
Output format
Default (human-readable):
QUERY: csrf middleware
----------------------------------------------------------------------
1. 5.51 django/core/checks/security/csrf.py
def _csrf_middleware():
return "django.middleware.csrf.CsrfViewMiddleware" in settings.MIDDLEWARE
2. 5.34 docs/howto/csrf.txt
...--json produces {"query": ..., "results": [{"path", "score", "snippet"}, ...]}.
Architecture
bm25.py CLI
├── resolve_corpus(spec) → local Path (downloads tarball if github.com/...)
├── cache_key(...) → 16-hex sha256 of inputs that determine the index
├── CorpusIndex.load(cache_dir) → returns cached index if present, else None
├── CorpusIndex.build(...) → walks files, tokenizes, indexes with bm25s
├── CorpusIndex.save(cache_dir) → persists to /home/claude/.bm25-cache/<key>/
├── query(q, k) → ranked (doc_idx, score) pairs
└── best_snippet(doc, q, lines) → pick line w/ most query-term hits + contextCache contents per directory:
bm25/— bm25s.BM25.save() output (NumPy arrays + vocab)corpus.pkl— pickled{paths, docs}so we can render snippets without re-reading the source filesmanifest.json— corpus root, files count, built_at timestamp
No network beyond optional tarball fetch on github.com/... corpora. No
state outside /home/claude/, which is ephemeral.
© oaustegard, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
Files
SKILL.md and 3 other files (scripts) in bm25 of oaustegard/claude-skills.
- SKILL.md
- CHANGELOG.md
- README.md
- scripts/bm25.py
Open the folder on GitHubat commit 90b0f1b
Compare with similar skills
Bm25 next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Bm25 this skilloaustegard/claude-skills | 150 | — | ~1.7k | Automated safety check: Pass | MIT | |
| Tests Query Corpusnetdata/netdata | 81k | — | ~5.3k | Automated safety check: Pass | GPL-3.0 | |
| Rank TrackingRyze-AI-Adgent/open-seo-mcp-skills | 4.7k | — | ~573 | Automated safety check: Pass | MIT | |
| Corpus Remeasurenubjs/nub | 4.4k | — | ~2.1k | Automated safety check: Pass | MIT | |
| Rank Trackeraaron-he-zhu/aaron-marketing-skills | 2.9k | 1 repos | ~2k | Automated safety check: Pass | Apache-2.0 | |
| RankHouseofmvps/claude-rank | 147 | — | ~510 | Automated safety check: Pass | MIT |
Similar skills
Rank Tracking
Ryze-AI-Adgent/open-seo-mcp-skills
Rank tracking without a tracker subscription — position movers between any two periods from real Search Console data.
Corpus Remeasure
nubjs/nub
The iteration cycle for the build-jail corpus — change the harness or nub, push it, re-measure ONE package@version on ONE platform, and read the answer.
Rank Tracker
aaron-he-zhu/aaron-marketing-skills
A skill your agent uses when the user asks to "track rankings" or "查排名"; measures keyword and SERP-position deltas over time from provided exports or connected tools, including AI-response checks.
Rank
Houseofmvps/claude-rank
Main orchestrator for claude-rank SEO/GEO/AEO toolkit. An agent skill from Houseofmvps/claude-rank.
- Official
Smart Contract Entry Point Analyzer
trailofbits/skills
Maps the state-changing entry points of a smart contract codebase and sorts them by access level, producing a structured audit report that leaves out read-only functions.
More from oaustegard/claude-skills
All 67 skills in this repoVega-Lite Interactive Charts
oaustegard/claude-skills
Builds interactive Vega-Lite charts from uploaded data: analyzes the fields, picks five to ten fitting chart types, and produces a React artifact with the data embedded inline.
Single-File HTML Composer
oaustegard/claude-skills
Builds self-contained single-file HTML pages such as reports, decks, postmortems, flowcharts and prototypes from a small spec using a bundled Python composer and templates.
Deciding With Confidence
oaustegard/claude-skills
Routes, triages, flags and rates a piece of text with a probability for every option: which department or queue a ticket goes to, which intent a message expresses, whether a yes/no condition holds…
Declauding
oaustegard/claude-skills
Rewrites model-sounding prose into plain technical writing and checks that every claim survives, for PR text, docs, commit messages and similar drafts.
Preact Developer
oaustegard/claude-skills
Guides building standards-based Preact apps with native-first choices, HTM syntax, import maps and vendored ESM, from single-file demos to larger builds.
Bluesky Zeitgeist Sampler
oaustegard/claude-skills
Deprecated sampler that captures short windows of the Bluesky firehose, clusters trending terms and builds an HTML report; replaced by the browsing-bluesky skill.
Questions about Bm25
What does Bm25 do?
Ranked content search over a text corpus you point it at, using BM25 (via xhluca/bm25s). Bm25 is an agent skill from oaustegard/claude-skills. Ranked content search over a text corpus you point it at, using BM25 (via xhluca/bm25s).
When should I use Bm25?
Bm25 fits situations like: rank these documents; search this corpus; which files are most about X; find content about Y.
How do I install Bm25 in Claude Code?
Run `npx skills add oaustegard/claude-skills --skill bm25 -a claude-code`. Or copy the skill folder (bm25 in oaustegard/claude-skills) into .claude/skills/bm25 in your project. Claude Code loads it when a task matches its description.
How do I install Bm25 in Codex?
Run `npx skills add oaustegard/claude-skills --skill bm25 -a codex`. Or copy the skill folder (bm25 in oaustegard/claude-skills) into .agents/skills/bm25 in your project. Codex loads it when a task matches its description.
Can I use Bm25 in Cursor, Gemini CLI or GitHub Copilot?
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add oaustegard/claude-skills --skill bm25 -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/bm25, .gemini/skills/bm25, .github/skills/bm25 and .opencode/skills/bm25 in your project.
What does Bm25 need to run?
Going by SKILL.md and its folder, Bm25 needs Python for the scripts in its folder, the command-line tools its instructions call (python3 and uv) and credentials named GH_TOKEN. Our summary lists: Python 3.
Does Bm25 access the network?
SKILL.md contains no URLs. Its commands use uv, which can reach the network depending on how they are called. This is read from the text; nothing was executed.
Is Bm25 safe to install?
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. The check reads SKILL.md only: the scripts in the folder are not scanned, so read them before running anything.
What licence does Bm25 use?
Bm25 is published under the MIT licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.
How many tokens does Bm25 use?
About 1.7k tokens (SKILL.md is roughly 7k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.
What are the alternatives to Bm25?
Skills that share tags, products or a category with Bm25: Tests Query Corpus (netdata/netdata, 81k stars), Rank Tracking (Ryze-AI-Adgent/open-seo-mcp-skills, 4.7k stars), Corpus Remeasure (nubjs/nub, 4.4k stars) and Rank Tracker (aaron-he-zhu/aaron-marketing-skills, 2.9k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
Who maintains Bm25?
oaustegard (a GitHub user) maintains it in oaustegard/claude-skills, which has 150 GitHub stars. The repository holds 67 skills in this directory. The repository was last updated on October 9, 2026.
Source: oaustegard/claude-skills on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.