PR Babysitter
openinterpreter/openinterpreter
Watches an open GitHub pull request until it merges, handling review comments, diagnosing CI failures and retrying flaky checks along the way.
Converts one or more GitHub Issues into standardized benchmark data using automated scripts.
MITAuto-check passed
Install Issue To Eval
$ npx skills add RConsortium/pharma-skills --skill issue-to-eval -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install RConsortium/pharma-skills issue-to-eval --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/RConsortium/pharma-skills.git skills-src && mkdir -p .claude/skills && cp -r skills-src/_automation/issue-to-eval .claude/skills/issue-to-eval && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "issue-to-eval" agent skill from https://github.com/RConsortium/pharma-skills/tree/main/_automation/issue-to-eval into .claude/skills/issue-to-eval/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "issue-to-eval", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/RConsortium/pharma-skills/tree/main/_automation/issue-to-evalType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add RConsortium/pharma-skills --skill issue-to-eval -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install RConsortium/pharma-skills issue-to-eval --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/RConsortium/pharma-skills.git skills-src && mkdir -p .agents/skills && cp -r skills-src/_automation/issue-to-eval .agents/skills/issue-to-eval && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "issue-to-eval" agent skill from https://github.com/RConsortium/pharma-skills/tree/main/_automation/issue-to-eval into .agents/skills/issue-to-eval/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "issue-to-eval", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add RConsortium/pharma-skills --skill issue-to-eval -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install RConsortium/pharma-skills issue-to-eval --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/RConsortium/pharma-skills.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/_automation/issue-to-eval .cursor/skills/issue-to-eval && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "issue-to-eval" agent skill from https://github.com/RConsortium/pharma-skills/tree/main/_automation/issue-to-eval into .cursor/skills/issue-to-eval/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "issue-to-eval", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/RConsortium/pharma-skills.git --path _automation/issue-to-eval--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add RConsortium/pharma-skills --skill issue-to-eval -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install RConsortium/pharma-skills issue-to-eval --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/RConsortium/pharma-skills.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/_automation/issue-to-eval .gemini/skills/issue-to-eval && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "issue-to-eval" agent skill from https://github.com/RConsortium/pharma-skills/tree/main/_automation/issue-to-eval into .gemini/skills/issue-to-eval/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "issue-to-eval", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install RConsortium/pharma-skills issue-to-evalInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add RConsortium/pharma-skills --skill issue-to-eval -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/RConsortium/pharma-skills.git skills-src && mkdir -p .github/skills && cp -r skills-src/_automation/issue-to-eval .github/skills/issue-to-eval && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "issue-to-eval" agent skill from https://github.com/RConsortium/pharma-skills/tree/main/_automation/issue-to-eval into .github/skills/issue-to-eval/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "issue-to-eval", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add RConsortium/pharma-skills --skill issue-to-eval -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install RConsortium/pharma-skills issue-to-eval --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/RConsortium/pharma-skills.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/_automation/issue-to-eval .opencode/skills/issue-to-eval && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "issue-to-eval" agent skill from https://github.com/RConsortium/pharma-skills/tree/main/_automation/issue-to-eval into .opencode/skills/issue-to-eval/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "issue-to-eval", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
Facts
- Skill name
issue-to-eval- GitHub stars
- 119
- Token cost
- ~463 tokens
- SKILL.md length
- 201 words
- Files
- 5 (incl. scripts)
- Skills in repo
- 13
- Repo updated
- First seen
- Licence
- MIT
At a glance
Converts one or more GitHub Issues into standardized benchmark data using automated scripts.
- A user provides an issue number
- SKILL.md covers Task Flow, Review Output and Requirements
- Runs Python scripts from its folder; calls python3
- URL and wants to add it to the evaluation suite
What it does
Issue To Eval is an agent skill from RConsortium/pharma-skills. Converts one or more GitHub Issues into standardized benchmark data using automated scripts. Use when a user provides an issue number or URL and wants to add it to the evaluation suite.
Its SKILL.md is about 460 tokens, which your agent loads only when the skill is triggered. The skill folder holds 5 other files, including scripts (for example `README.md`, `scripts/import_issue_eval.py` and `scripts/sync_benchmarks.py`).
It works with GitHub. The repository describes itself as: A collection of agent skills for BioPharma use cases GSDBench Intake https://rconsortium.github.io/pharma-skills/gsdbench-intake/. The licence is MIT.
When your agent uses it
- A user provides an issue number
- URL and wants to add it to the evaluation suite
Example prompts
- “Use the issue-to-eval skill to convert one or more GitHub Issues into standardized benchmark data using automated scripts”
- “/issue-to-eval”
Requirements
- Python 3
What it can do on your machine
Read from SKILL.md and the folder at commit ae5d83b. It shows what the files ask for, not the result of running them.
Tool permissions
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Runs code
Ships 2 files in scripts/ (Python), which the agent can run.
Shell commands in SKILL.md call:
python3
From the folder's file list and the shell code blocks in SKILL.md.
Network
No URLs in SKILL.md.
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Context cost
Issue To Eval loads about 463 tokens when it runs. Until then it costs about 50 tokens; SKILL.md has 201 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
Safety
Auto-check passed
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.
SKILL.md
The full file from RConsortium/pharma-skills at commit ae5d83b, republished under its MIT licence (© RConsortium). 201 words, ~463 tokens.
Download SKILL.mdSave it as
.claude/skills/issue-to-eval/SKILL.md (or your agent's skills folder). This skill also uses 4 other files; get the full folder from GitHub.- name
- issue-to-eval
- description
- Converts one or more GitHub Issues into standardized benchmark data using automated scripts. Use when a user provides an issue number or URL and wants to add it to the evaluation suite.
Issue to Eval
Converts GitHub Issues into standardized benchmark evaluation cases (JSON) and saves them to the _automation/evals/ directory. It automatically discovers new issues labeled benchmark and updates existing evaluations if the issue content has been modified.
Task Flow
Option A: Sync All Benchmarks (Recommended)
Use this to automatically identify and update all evaluations from GitHub:
python3 _automation/issue-to-eval/scripts/sync_benchmarks.py- Discovers all issues with the
benchmarklabel. - Compares each issue against the local files in
evals/. - Adds new cases and updates existing ones if the prompt or assertions have changed.
Option B: Import Specific Issue
Use this for a one-off import or testing:
python3 _automation/issue-to-eval/scripts/import_issue_eval.py --issue {ISSUE_NUMBER_OR_URL}Review Output
- Report the status for each issue (Success/Updated/Skipped/Error) to the user.
- If an issue is modified on GitHub, running the sync script will propagate those changes to the local evaluation suite.
Requirements
- The issue MUST follow the standard benchmark template with headers:
## Skills, optional## Language,## Query,## Expected Output,## Attached Files / Input Context, and## Rubric Criteria (Assertions). ## Skillsmay contain one or more skill names, one per line or comma-separated. The generated eval stores them intarget_skills.## Languageis optional. When present, the generated eval stores it inlanguageso the benchmark runner can pass the same language constraint to both agents.
© RConsortium, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
Files
SKILL.md and 4 other files (scripts) in _automation/issue-to-eval of RConsortium/pharma-skills.
- SKILL.md
- LICENSE
- README.md
- scripts/import_issue_eval.py
- scripts/sync_benchmarks.py
Open the folder on GitHubat commit ae5d83b
Compare with similar skills
Issue To Eval next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Issue To Eval this skillRConsortium/pharma-skills | 119 | — | ~463 | Automated safety check: Pass | MIT | |
| PR Babysitteropeninterpreter/openinterpreter | 69k | 3 repos | ~4.2k | Automated safety check: Pass | Apache-2.0 | |
| Diagnosing Superpowers Sessionsobra/superpowers | 297k | 3 repos | ~1.7k | Automated safety check: Pass | MIT | |
| Greplooponyx-dot-app/onyx | 32k | 4 repos | ~3.3k | Automated safety check: Pass | MIT | |
| GitHub Deep Researchbytedance/deer-flow | 84k | 4 repos | ~1.3k | Automated safety check: Pass | MIT | |
| Update V8 Versionopeninterpreter/openinterpreter | 69k | 2 repos | ~845 | Automated safety check: Pass | Apache-2.0 |
Similar skills
Diagnosing Superpowers Sessions
obra/superpowers
Investigates a session where Superpowers went wrong, reads the transcripts on disk and produces an evidence-cited report, optionally prepared as a bug report for the maintainers.
Greploop
onyx-dot-app/onyx
Iteratively improves a PR (GitHub), MR (GitLab), or shelved changelist (Perforce) until Greptile gives it a 5/5 confidence score with zero unresolved comments.
GitHub Deep Research
bytedance/deer-flow
Researches a GitHub repository over four rounds using the GitHub API and web search, then writes a structured markdown report with timeline, metrics and Mermaid diagrams.
Update V8 Version
openinterpreter/openinterpreter
Bumps the pinned v8 and rusty_v8 versions in Codex, validates the release-candidate path with the v8-canary check, and traces failures to upstream build changes.
Last30days
mvanhorn/last30days-skill
Research what people actually say about any topic in the last 30 days.
More from RConsortium/pharma-skills
All 13 skills in this repoRounding
RConsortium/pharma-skills
Audit R code that prepares CSR/TLF statistics for SAS-compatible rounding compliance (ties away from zero, round-once-at-display, fixed trailing-zero precision).
Weekly Summary
RConsortium/pharma-skills
Generate a concise weekly progress summary for the pharmaskills repository.
Admiral Adae
RConsortium/pharma-skills
Derives an ADaM Adverse Events Analysis Dataset (ADAE) using the {admiral} R package and pharmaverse ecosystem.
Admiral Adsl
RConsortium/pharma-skills
Derives an ADaM Subject-Level Analysis Dataset (ADSL) using the {admiral} R package and pharmaverse ecosystem.
Admiral Bds
RConsortium/pharma-skills
Derives ADaM Basic Data Structure (BDS) datasets using the {admiral} R package.
Sdtm Oak
RConsortium/pharma-skills
Derives CDISC SDTM domains from raw clinical (EDC/eCRF) data using the {sdtm.oak} R package.
Related
Works with
Questions about Issue To Eval
What does Issue To Eval do?
Converts one or more GitHub Issues into standardized benchmark data using automated scripts. Issue To Eval is an agent skill from RConsortium/pharma-skills. Converts one or more GitHub Issues into standardized benchmark data using automated scripts.
When should I use Issue To Eval?
Issue To Eval fits situations like: A user provides an issue number; URL and wants to add it to the evaluation suite.
How do I install Issue To Eval in Claude Code?
Run `npx skills add RConsortium/pharma-skills --skill issue-to-eval -a claude-code`. Or copy the skill folder (_automation/issue-to-eval in RConsortium/pharma-skills) into .claude/skills/issue-to-eval in your project. Claude Code loads it when a task matches its description.
How do I install Issue To Eval in Codex?
Run `npx skills add RConsortium/pharma-skills --skill issue-to-eval -a codex`. Or copy the skill folder (_automation/issue-to-eval in RConsortium/pharma-skills) into .agents/skills/issue-to-eval in your project. Codex loads it when a task matches its description.
Can I use Issue To Eval in Cursor, Gemini CLI or GitHub Copilot?
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add RConsortium/pharma-skills --skill issue-to-eval -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/issue-to-eval, .gemini/skills/issue-to-eval, .github/skills/issue-to-eval and .opencode/skills/issue-to-eval in your project.
What does Issue To Eval need to run?
Going by SKILL.md and its folder, Issue To Eval needs Python for the scripts in its folder and the command-line tools its instructions call (python3). Our summary lists: Python 3.
Does Issue To Eval access the network?
SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.
Is Issue To Eval safe to install?
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. The check reads SKILL.md only: the scripts in the folder are not scanned, so read them before running anything.
What licence does Issue To Eval use?
Issue To Eval is published under the MIT licence (from the LICENSE file in the skill folder). It allows redistribution, so the full SKILL.md is shown on this page.
How many tokens does Issue To Eval use?
About 463 tokens (SKILL.md is roughly 1.9k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.
What are the alternatives to Issue To Eval?
Skills that share tags, products or a category with Issue To Eval: PR Babysitter (openinterpreter/openinterpreter, 69k stars), Diagnosing Superpowers Sessions (obra/superpowers, 297k stars), Greploop (onyx-dot-app/onyx, 32k stars) and GitHub Deep Research (bytedance/deer-flow, 84k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
Who maintains Issue To Eval?
RConsortium (a GitHub organization) maintains it in RConsortium/pharma-skills, which has 119 GitHub stars. The repository holds 13 skills in this directory. The repository was last updated on October 4, 2026.
Source: RConsortium/pharma-skills on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.