Benchmark Update

Context

![ -f pyproject.toml ] && grep '^name' pyproject.toml | head -1 || echo "No pyproject.toml" ![ -f docs/benchmark.md ] && head -15 docs/benchmark.md || echo "No docs/benchmark.md yet" ![ -f docs/benchmark.md ] && grep -c '^|' docs/benchmark.md || echo 0 ![ -f CLAUDE.md ] && grep -i 'ordering\|benchmark\|comparison' CLAUDE.md | head -5 || echo "No ordering conventions found"

Instructions

You create and maintain docs/benchmark.md, a competitive benchmark page comparing the current project against alternatives in the same space. The page follows a standard template with feature tables, GitHub activity badges, popularity charts, distribution info, and metadata.

Reference examples

Fetch these files as reference when building or auditing a benchmark page:

• kdeldycke/meta-package-manager/docs/benchmark.md: package manager comparison with features, operations, OS support, distribution, activity, popularity, and metadata tables.
• kdeldycke/click-extra/docs/benchmark.md: CLI framework comparison with DX/UX feature tables, parser flag scoping, unique strengths, gaps with upstream issue references, and an "Excluded frameworks" section.

Scope selection

• audit (default): Run all checks on an existing benchmark page and report findings. Do not edit unless the user confirms.
• init: Create a benchmark page from scratch. Research competitors, build feature tables, and populate all sections.
• add <project>: Research a specific project and add it to all tables in the correct position.
• refresh-badges: Verify all badge URLs resolve. Fix broken ones (repos that moved, renamed packages).

Column ordering convention

If CLAUDE.md defines a benchmark ordering convention, follow it. Otherwise, place the current project first, its direct dependencies/foundation second, then remaining projects sorted by GitHub stars (descending).

Creating a benchmark from scratch (init)

1. Identify the project and its domain

Read pyproject.toml for the project name, description, and keywords. Determine what category of software this project is (CLI framework, package manager, static site generator, linter, etc.).

2. Discover competitors

Search for alternatives in the same space:

• Search GitHub: gh search repos "<domain keywords>" --sort stars --limit 30
• Search PyPI and other registries for similar tools.
• Check "awesome" lists on GitHub.
• Look at the project's own README or docs for mentions of alternatives.

For each candidate, collect: name, GitHub owner/repo, approximate star count, one-line description, and language/ecosystem.

Present the candidates and ask the user which to include.

3. Research features

For each included project, research its feature set by reading its documentation, README, and changelog. Build comparison tables relevant to the domain. Common table categories:

• Features: core capabilities, grouped by audience (developer experience vs. end-user experience) when applicable.
• Activity: GitHub badges (watchers, contributors, commit activity, commits since latest release, last release date, last commit, open issues, open PRs, forks, dependencies freshness).
• Popularity: star history chart + badges (stars, SourceRank, dependent repos).
• Distribution: package registry badges (PyPI, crates.io, npm, Homebrew, etc.) as applicable.
• Metadata: license, main language, latest version, benchmark date.

Use ✅ for a supported feature, 🟡 for partial or opt-in support, ❌ for a feature the project lacks or rejects, and N/A where the row does not apply. Link every cell to its evidence: the documentation or a source line for ✅ and 🟡, and for ❌ a maintainer statement, a request closed as not planned or left open, or a documented limit. Absence of the feature is not evidence: with no citable source, leave the cell blank.

4. Gap analysis

After building the feature tables, analyze them to identify:

• Unique strengths: features where this project leads or is the only one offering the capability. Summarize each with a brief explanation of why it matters.
• Gaps and opportunities: features where competitors lead and this project could improve. For each gap:
  • Search the project's primary upstream dependency (if any) for related open issues and PRs using gh issue list --repo <upstream> --search "<feature>" --state open.
  • Link to specific upstream issues with their number, title, and demand signals (thumbs-up count, comment count).
  • Note whether upstream has declined the feature, has a pending PR, or has never been asked.
  • Describe what this project could do to close the gap (override, extend, integrate a third-party package, etc.).

This analysis turns the benchmark from a static comparison into an actionable roadmap.

5. Build the page

Follow this template structure:

# {octicon}`trophy` Benchmark

<intro paragraph about why this comparison exists>

## <Feature category 1>

<legend if needed>

| Feature | `this-project` | `competitor-1`[^1] | ... |
| ... | :---: | :---: | ... |

<brief prose highlighting key differences>

## <Feature category 2>

...

## Unique strengths

<bullet list of features where this project leads, with brief explanations>

## Gaps and opportunities

### <Gap title>

<description of the gap, with links to upstream issues like `[owner/repo#N](url)` including demand signals>

<what this project could do to close the gap>

### <Gap title>

...

## Activity

| Metrics | `this-project` | `competitor-1`[^1] | ... |
(GitHub badge rows: watchers, contributors, commit activity, etc.)

## Popularity

![Star history of `this-project` and its alternatives](assets/star-history-compared.svg)

<a sentence on the axis, when the chart is logarithmic, and one on how far back the store reaches>

| Metrics | `this-project` | `competitor-1`[^1] | ... |
(Stars, SourceRank, Dependent repos)

## Distribution

| Registry | `this-project` | `competitor-1`[^1] | ... |
(PyPI, crates.io, npm, Homebrew downloads)

## Metadata

| Metadata | `this-project` | `competitor-1`[^1] | ... |
(License, main language, latest version, benchmark date)

## Excluded projects

~~~{note}
<Project name> is not included because <reason>.
~~~

## Project URLs

[^1]: [<url>](<url>)

Badge format follows shields.io conventions: ![GitHub](https://img.shields.io/github/<metric>/<owner>/<repo>?label=%20&style=flat-square) for compact badges with no label text.

Star history chart: render it locally, never as a third-party embed. GitHub restricted its stargazer endpoints to a repository's own admins in June 2026, which left every third-party star chart rendering an error card. It reopened an anonymous star-history endpoint in September 2026. A history committed to the repository cannot be revoked upstream, whatever GitHub changes next. repomatic sample-metrics records the counts into a committed CSV and draws them as a themeable SVG: declare the projects in [tool.repomatic.metrics] subjects, declare a chart in [tool.repomatic.metrics] charts, and reference that chart's output path from this section. A comparison spanning projects of very different sizes wants scale = "logarithmic", and one comparing trajectories rather than dates wants mode = "relative".

A GitHub peer gets its whole curve, rebuilt from that endpoint back to its first star. That curve counts only the stars the repository still holds, so it understates every past week by the stars withdrawn since. A peer on any other forge is sampled forward, one reading per run, and that history cannot be backdated: it carries only two points at first, its creation date and the first reading, so a chart drawn early states a straight line between them. Both are worth a sentence under the chart rather than hiding.