Agent skill

Open-Science Skill Creator

by aipoch in aipoch/open-science

Creates, revises, evaluates and publishes skills in the Open-Science app through its native host.skills composer, with optional test prompts and benchmarks.

Apache-2.0Auto-check passedAgent Workflows

Install Open-Science Skill Creator

skills CLI
$ npx skills add aipoch/open-science --skill skill-creator -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install aipoch/open-science skill-creator --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/aipoch/open-science.git skills-src && mkdir -p .claude/skills && cp -r skills-src/resources/skills/skill-creator .claude/skills/skill-creator && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
skill-creator
GitHub stars
5.5k
Token cost
~1.7k tokens
SKILL.md length
829 words
Files
17 (incl. scripts, references, assets)
Skills in repo
8
Repo updated
First seen
Licence
Apache-2.0

At a glance

Creates, revises, evaluates and publishes skills in the Open-Science app through its native host.skills composer, with optional test prompts and benchmarks.

  • Works in 6 steps: Capture intent and examples. → Draft or revise the Skill. → Review the package with the user. → …
  • Creating a reusable workflow as a skill in the Open-Science app
  • SKILL.md covers Native composer, Choose the current stage, Capture intent and Author the package, plus 4 more sections
  • Runs JavaScript scripts from its folder

What it does

This skill creates, revises, evaluates and publishes skills for the Open-Science app through its native `host.skills` composer, driven from a JavaScript control-plane REPL. Skills are application-managed packages rather than artifacts. The composer lists, reads, edits, publishes and deletes skills: an edit without `oldString` creates a file and fails if it exists, with `oldString` the old text must occur exactly once, and an existing draft is never silently overwritten. Publishing promotes the whole draft into Personal Skills, and deletion always needs app approval.

The agent infers the current stage and starts there: capture intent and examples, draft or revise, review with you, optionally evaluate realistic prompts, improve from the evidence, and publish only after you accept the draft. Evaluation is not forced, since subjective or exploratory skills may be better reviewed in conversation. Built-in or imported skills are forked into a personal copy under a new lowercase hyphenated name. Bundled helpers include grader, analyzer and comparator agent notes, an eval viewer and scripts to run evals, aggregate benchmarks, validate skills and improve descriptions.

When your agent uses it

  • Creating a reusable workflow as a skill in the Open-Science app
  • Revising an existing personal skill
  • Running test prompts or benchmarks against a skill
  • Improving a skill's trigger description

Example prompts

  • “Turn my literature screening steps into a reusable skill.”
  • “Fork the built-in plotting skill and add a rule for colorblind-safe palettes.”
  • “Run a benchmark on my sequence-alignment skill with a few realistic prompts.”
  • “My skill never triggers on short requests. Improve its description.”

Requirements

  • The Open-Science desktop app, which provides the host.skills composer

Workflow steps

6 steps, taken from the first numbered list in SKILL.md.

  1. Capture intent and examples.
  2. Draft or revise the Skill.
  3. Review the package with the user.
  4. Optionally evaluate realistic prompts.
  5. Improve from evidence and repeat.
  6. Publish only after the user accepts the draft.

What it can do on your machine

Read from SKILL.md and the folder at commit 51d7079. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Ships 7 files in scripts/ (JavaScript, from the files we listed), which the agent can run.

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    No URLs in SKILL.md.

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Open-Science Skill Creator loads about 1.7k tokens when it runs, and up to ~2.6k if it reads all its reference files. Until then it costs about 68 tokens; SKILL.md has 829 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~68
When it runs · the whole SKILL.md, loaded when a task matches
~1.7k
With references · SKILL.md plus every file in references/, read only if the agent opens them
~2.6k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.

SKILL.md

The full file from aipoch/open-science at commit 51d7079, republished under its Apache-2.0 licence (© aipoch). 829 words, ~1,662 tokens.

Download SKILL.mdSave it as .claude/skills/skill-creator/SKILL.md (or your agent's skills folder). This skill also uses 16 other files; get the full folder from GitHub.
name
skill-creator
description
Create, revise, evaluate, publish, and improve Open-Science Skills through the native JavaScript host.skills composer. Use when the user wants a reusable workflow, an existing Skill changed, test cases or benchmarks for a Skill, or better Skill triggering.

Skill Creator

Create one focused, reusable Skill package. Skills are application-managed packages, not Artifacts. Use the JavaScript control-plane REPL and the native host.skills composer for lifecycle operations.

Native composer

javascript
await host.skills.list()
await host.skills.read(name)
await host.skills.read(name, path)
await host.skills.validate(name)
await host.skills.edit(name, path, content)
await host.skills.edit(name, path, replacement, oldString)
await host.skills.publish(name)
await host.skills.publish(name, true)
await host.skills.delete(stableId)

Without oldString, edit creates a file and fails if it exists. With oldString, the old text must occur exactly once. Never silently overwrite an existing draft file. publish promotes the complete draft into Personal Skills. delete is privileged and always uses app approval. When a published Skill and its draft coexist, delete only by the exact draft-<name> or personal-<name> id returned from list(); never guess from the shared display name.

Choose the current stage

Infer where the user is in the workflow and start there:

  1. Capture intent and examples.
  2. Draft or revise the Skill.
  3. Review the package with the user.
  4. Optionally evaluate realistic prompts.
  5. Improve from evidence and repeat.
  6. Publish only after the user accepts the draft.

Do not force evaluation. Objectively verifiable workflows benefit from test cases; subjective writing or exploratory Skills may be better reviewed directly in conversation.

Capture intent

Extract what is already known from the conversation before asking questions. Confirm only gaps that materially change behavior:

  • What should the Skill enable an agent to do?
  • When should it trigger, including near-miss cases where it should not?
  • What inputs and output formats matter?
  • What counts as success, and what failures need explicit handling?
  • Are scripts, references, assets, connectors, or example files required?
  • Does the user want test cases now?

Prefer one concise question at a time. Calibrate terms such as benchmark, JSON, or assertion to the user's technical comfort.

Author the package

  1. Call host.skills.list() before editing. Read every existing file you intend to change.
  2. For Built-in or Imported Skills, create a Personal fork under a new lowercase hyphenated name.
  3. Use frontmatter with name and description, plus optional displayName; name is the immutable safe draft name and defaults as the presentation label when displayName is omitted.
  4. Put stable procedures in SKILL.md, detailed knowledge in references/, deterministic automation in scripts/, and output templates in assets/.
  5. Prefer imperative instructions and explain why constraints matter. Avoid brittle lists of MUSTs.
  6. Keep SKILL.md focused. Link directly to optional resources and state when to read them.
  7. Re-read changed files, call host.skills.validate(name), and show the user the important behavior and boundaries before publishing.

Do not promise automatic kernel sidecars, per-Specialist environments, or connector tool patterns; those capabilities are not part of the current composer.

Create test cases

When the user wants evaluation, propose two or three realistic prompts. Ask them to confirm or revise the set before running anything. Store output-evaluation cases as evals/evals.json. Store trigger and near-miss cases as trigger-evals.json. Follow references/schemas.md.

Good cases cover different phrasings, input shapes, edge cases, and near misses. Expectations should be observable from the transcript or output files. Use human review for qualities that cannot be reliably reduced to assertions.

Show full SKILL.md (344 more words)Show less

Run and evaluate

Evaluation is capability-gated. First check whether this runtime exposes host.skills.evals. If it does not, run a qualitative sanity check in the current conversation or publish without evaluation if the user chooses; never claim that baseline, blind, or trigger evaluation ran when it did not.

When host.skills.evals is available:

  1. Freeze the draft identity and revision.
  2. Create paired runs for each case: one with the Skill and one baseline.
  3. Keep inputs, provider/model, and tool capabilities equal across the pair.
  4. Save outputs, transcript, timing, token counts, and actual Skill activity.
  5. Grade expectations using agents/grader.md.
  6. Aggregate results with scripts/aggregate-benchmark.js.
  7. Generate the review page with eval-viewer/generate-review.js and let the user review outputs before changing the Skill.
  8. Use agents/comparator.md only when A/B origins are genuinely hidden.
  9. Use agents/analyzer.md to explain benchmark patterns and comparison results.

Never use persistent host.agents Specialists as pretend isolated evaluators. Never start another provider CLI from the REPL to bypass the app-owned Session and approval boundaries.

Improve from feedback

Read user feedback, grades, transcripts, and benchmark notes together. Generalize from repeated failures instead of overfitting to one prompt. Look for:

  • ambiguous instructions that led to divergent behavior;
  • repeated helper code that belongs in scripts/;
  • expectations that pass both configurations and therefore do not measure Skill value;
  • flaky cases with high variance;
  • time or token costs that outweigh the quality gain;
  • false-positive and false-negative trigger cases.

scripts/improve-description.js can build and parse a description-improvement prompt, but the current Agent or an app-owned evaluation Session must perform the model call. Always show description changes and scores to the user before applying an exact-match edit.

Review and publish

Summarize the final behavior, boundaries, files, and any unverified capability. Publish with await host.skills.publish(name). Use overwrite = true only after the user explicitly chooses to replace an existing Personal Skill. Read the published SKILL.md back and report its actual id and origin.

If the user asks to attach it to a Specialist, read the live Specialist and Skill catalogs first, then call host.agents.attachSkill(...) and report the returned read-back. Never attach automatically.

© aipoch, Apache-2.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 16 other files (scripts, references, assets) in resources/skills/skill-creator of aipoch/open-science.

  • SKILL.md
  • agents/analyzer.md
  • agents/comparator.md
  • agents/grader.md
  • assets/eval_review.html
  • eval-viewer/generate-review.js
  • eval-viewer/viewer.html
  • package.json
  • references/schemas.md
  • scripts/aggregate-benchmark.js
  • scripts/generate-report.js
  • scripts/improve-description.js
  • scripts/index.js
  • scripts/quick-validate.js
  • scripts/run-eval.js
  • scripts/run-loop.js
  • … and 1 more

Open the folder on GitHubat commit 51d7079

Compare with similar skills

Open-Science Skill Creator next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Open-Science Skill Creator compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Open-Science Skill Creator this skillaipoch/open-science5.5k—~1.7kAutomated safety check: PassApache-2.0
Darwin Skill Optimizeralchaincyf/darwin-skill6.2k1 repos~4.7kAutomated safety check: PassMIT
Skill Quality ReviewerGalaxy-Dawn/claude-scholar5.7k1 repos~3kAutomated safety check: PassMIT
OpenCode Skill Creatorantongulin/opencode-skill-creator172—~8.1kAutomated safety check: PassApache-2.0
Skill Release Gaterohitg00/ai-engineering-from-scratch66k—~1kAutomated safety check: PassMIT
Skill Creatorzhayujie/CowAgent47k—~4.7kAutomated safety check: NotesMIT

Similar skills

  • Darwin Skill Optimizer

    alchaincyf/darwin-skill

    Scores SKILL.md files on a nine-dimension rubric, then improves them in a keep-or-revert loop with independent judge agents, test prompts, git history and human checkpoints.

    6.2k GitHub starsUsed in 1 repo~4.7k tokens
    Agent WorkflowsAuto-check passed
  • Skill Quality Reviewer

    Galaxy-Dawn/claude-scholar

    Scores a skill across description, content organization, writing style and structure, then produces letter grades and a prioritized improvement plan.

    5.7k GitHub starsUsed in 1 repo~3k tokens
    Agent WorkflowsAuto-check passed
  • OpenCode Skill Creator

    antongulin/opencode-skill-creator

    Walks you through drafting, testing, evaluating and tuning a skill for OpenCode, from an intake interview to description optimization.

    172 GitHub stars~8.1k tokensUpdated 7 days ago
    Agent WorkflowsAuto-check passed
  • Skill Release Gate

    rohitg00/ai-engineering-from-scratch

    Evaluates an Agent Skill bundle before release for structure, trigger quality, artifact improvement, script correctness, safety, installed-tree integrity and host portability.

    66k GitHub stars~1k tokensUpdated 2 days ago
    Agent WorkflowsAuto-check passed
  • Skill Creator

    zhayujie/CowAgent

    Guides creating, installing and updating agent skills in a workspace: SKILL.md frontmatter, bundled scripts and references, with scripts to scaffold, validate and package.

    47k GitHub stars~4.7k tokensUpdated today
    Agent WorkflowsAuto-check: notes
  • Skill Judge

    shareAI-lab/Kode-CLI

    Evaluates the design quality of an agent skill against official specifications and patterns from existing examples, scoring it and suggesting improvements.

    5.2k GitHub starsUsed in 4 repos~7.5k tokens
    Agent WorkflowsAuto-check passed

More from aipoch/open-science

All 8 skills in this repo
  • Compute Environment Setup

    aipoch/open-science

    Prepares setup instructions and a named activation file for a user-managed software environment on an Open-Science SSH or Slurm compute host.

    5.5k GitHub starsUsed in 1 repo~2.6k tokens
    Auto-check passed
  • Handles /Customize requests by sending Skill work to the internal skill-creator and managing Specialist agents through the JavaScript host.agents SDK.

    5.5k GitHub stars~2.7k tokensUpdated today
    Auto-check passed
  • Figure Style

    aipoch/open-science

    Publication-grade correctness and legibility rules for final-deliverable scientific figures, not exploratory plots.

    5.5k GitHub stars~5.1k tokensUpdated today
    Auto-check passed
  • Paper Narrative

    aipoch/open-science

    Judge and reshape the story told by an entire paper figure deck.

    5.5k GitHub stars~4.4k tokensUpdated today
    Auto-check passed
  • Open-Science Host Inspection

    aipoch/open-science

    Teaches an agent to inspect Open-Science's JavaScript control REPL, check which host.* calls are allowed, and find project files, sessions and agent frames.

    5.5k GitHub stars~3.8k tokensUpdated today
    Auto-check passed
  • Figure Composer

    aipoch/open-science

    Compose one publication-grade multi-panel figure. An agent skill from aipoch/open-science.

    5.5k GitHub stars~2.9k tokensUpdated today
    Auto-check passed

Works with

Categories

Questions about Open-Science Skill Creator

What does Open-Science Skill Creator do?

Creates, revises, evaluates and publishes skills in the Open-Science app through its native host.skills composer, with optional test prompts and benchmarks. skills` composer, driven from a JavaScript control-plane REPL. Skills are application-managed packages rather than artifacts.

When should I use Open-Science Skill Creator?

Open-Science Skill Creator fits situations like: creating a reusable workflow as a skill in the Open-Science app; revising an existing personal skill; running test prompts or benchmarks against a skill; improving a skill's trigger description.

How do I install Open-Science Skill Creator in Claude Code?

Run `npx skills add aipoch/open-science --skill skill-creator -a claude-code`. Or copy the skill folder (resources/skills/skill-creator in aipoch/open-science) into .claude/skills/skill-creator in your project. Claude Code loads it when a task matches its description.

How do I install Open-Science Skill Creator in Codex?

Run `npx skills add aipoch/open-science --skill skill-creator -a codex`. Or copy the skill folder (resources/skills/skill-creator in aipoch/open-science) into .agents/skills/skill-creator in your project. Codex loads it when a task matches its description.

Can I use Open-Science Skill Creator in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add aipoch/open-science --skill skill-creator -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/skill-creator, .gemini/skills/skill-creator, .github/skills/skill-creator and .opencode/skills/skill-creator in your project.

What does Open-Science Skill Creator need to run?

Going by SKILL.md and its folder, Open-Science Skill Creator needs JavaScript for the scripts in its folder. Our summary lists: The Open-Science desktop app, which provides the host.skills composer.

Does Open-Science Skill Creator access the network?

SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.

Is Open-Science Skill Creator safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. The check reads SKILL.md only: the scripts in the folder are not scanned, so read them before running anything.

What licence does Open-Science Skill Creator use?

Open-Science Skill Creator is published under the Apache-2.0 licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Open-Science Skill Creator use?

About 1.7k tokens (SKILL.md is roughly 6.6k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 956 tokens, read only when the agent opens those files.

What are the alternatives to Open-Science Skill Creator?

Skills that share tags, products or a category with Open-Science Skill Creator: Darwin Skill Optimizer (alchaincyf/darwin-skill, 6.2k stars), Skill Quality Reviewer (Galaxy-Dawn/claude-scholar, 5.7k stars), OpenCode Skill Creator (antongulin/opencode-skill-creator, 172 stars) and Skill Release Gate (rohitg00/ai-engineering-from-scratch, 66k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Open-Science Skill Creator?

aipoch (a GitHub organization) maintains it in aipoch/open-science, which has 5,475 GitHub stars. The repository holds 8 skills in this directory. The repository was last updated on October 8, 2026.

Source: aipoch/open-science on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.