Prompt Improver
severity1/claude-code-prompt-improver
This skill enriches vague prompts with targeted research and clarification before execution.
A skill your agent uses when designing or auditing the empirical core of a COLM paper — contamination analysis for evaluation data, fair baselines under matched prompting and compute, pinned model…
Install Colm Experiments
$ npx skills add brycewang-stanford/Awesome-Journal-Skills --skill colm-experiments -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install brycewang-stanford/Awesome-Journal-Skills colm-experiments --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/brycewang-stanford/Awesome-Journal-Skills.git skills-src && mkdir -p .claude/skills && cp -r skills-src/COLM-Skills/skills/colm-experiments .claude/skills/colm-experiments && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "colm-experiments" agent skill from https://github.com/brycewang-stanford/Awesome-Journal-Skills/tree/main/COLM-Skills/skills/colm-experiments into .claude/skills/colm-experiments/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "colm-experiments", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/brycewang-stanford/Awesome-Journal-Skills/tree/main/COLM-Skills/skills/colm-experimentsType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add brycewang-stanford/Awesome-Journal-Skills --skill colm-experiments -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install brycewang-stanford/Awesome-Journal-Skills colm-experiments --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/brycewang-stanford/Awesome-Journal-Skills.git skills-src && mkdir -p .agents/skills && cp -r skills-src/COLM-Skills/skills/colm-experiments .agents/skills/colm-experiments && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "colm-experiments" agent skill from https://github.com/brycewang-stanford/Awesome-Journal-Skills/tree/main/COLM-Skills/skills/colm-experiments into .agents/skills/colm-experiments/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "colm-experiments", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add brycewang-stanford/Awesome-Journal-Skills --skill colm-experiments -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install brycewang-stanford/Awesome-Journal-Skills colm-experiments --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/brycewang-stanford/Awesome-Journal-Skills.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/COLM-Skills/skills/colm-experiments .cursor/skills/colm-experiments && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "colm-experiments" agent skill from https://github.com/brycewang-stanford/Awesome-Journal-Skills/tree/main/COLM-Skills/skills/colm-experiments into .cursor/skills/colm-experiments/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "colm-experiments", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/brycewang-stanford/Awesome-Journal-Skills.git --path COLM-Skills/skills/colm-experiments--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add brycewang-stanford/Awesome-Journal-Skills --skill colm-experiments -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install brycewang-stanford/Awesome-Journal-Skills colm-experiments --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/brycewang-stanford/Awesome-Journal-Skills.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/COLM-Skills/skills/colm-experiments .gemini/skills/colm-experiments && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "colm-experiments" agent skill from https://github.com/brycewang-stanford/Awesome-Journal-Skills/tree/main/COLM-Skills/skills/colm-experiments into .gemini/skills/colm-experiments/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "colm-experiments", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install brycewang-stanford/Awesome-Journal-Skills colm-experimentsInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add brycewang-stanford/Awesome-Journal-Skills --skill colm-experiments -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/brycewang-stanford/Awesome-Journal-Skills.git skills-src && mkdir -p .github/skills && cp -r skills-src/COLM-Skills/skills/colm-experiments .github/skills/colm-experiments && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "colm-experiments" agent skill from https://github.com/brycewang-stanford/Awesome-Journal-Skills/tree/main/COLM-Skills/skills/colm-experiments into .github/skills/colm-experiments/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "colm-experiments", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add brycewang-stanford/Awesome-Journal-Skills --skill colm-experiments -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install brycewang-stanford/Awesome-Journal-Skills colm-experiments --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/brycewang-stanford/Awesome-Journal-Skills.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/COLM-Skills/skills/colm-experiments .opencode/skills/colm-experiments && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "colm-experiments" agent skill from https://github.com/brycewang-stanford/Awesome-Journal-Skills/tree/main/COLM-Skills/skills/colm-experiments into .opencode/skills/colm-experiments/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "colm-experiments", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
Facts
- Skill name
colm-experiments- GitHub stars
- 1.2k
- Token cost
- ~1.6k tokens
- SKILL.md length
- 774 words
- Files
- 1
- Skills in repo
- 2,387
- Repo updated
- First seen
- Licence
- MIT
At a glance
A skill your agent uses when designing or auditing the empirical core of a COLM paper — contamination analysis for evaluation data, fair baselines under matched prompting and compute, pinned model…
- Auditing the empirical core of a COLM paper — contamination analysis for evaluation data
- SKILL.md covers The four LM-specific attacks…, Uncertainty that matches the…, One config per number and Compute disclosure, plus 4 more sections
- Instructions only: no scripts, shell commands, URLs or credentials in SKILL.md
- Fair baselines under matched prompting and compute
What it does
Colm Experiments is an agent skill from brycewang-stanford/Awesome-Journal-Skills. Use when designing or auditing the empirical core of a COLM paper — contamination analysis for evaluation data, fair baselines under matched prompting and compute, pinned model versions and decoding parameters, uncertainty over runs and samples, scaling coverage, and honest reporting of API-model comparisons.
Its SKILL.md is about 1.6k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.
It sits in AI & LLM Engineering, covering Prompt engineering. The repository describes itself as: Journal-specific Claude Code/Codex skill packs covering mainstream journals — AER, QJE, Nature, Cell, 管理世界, 经济研究 & 200+ more — your fast track to getting published. ｜ 覆盖主流期刊的… The licence is MIT.
When your agent uses it
- Auditing the empirical core of a COLM paper — contamination analysis for evaluation data
- Fair baselines under matched prompting and compute
- Pinned model versions and decoding parameters
- Uncertainty over runs and samples
Example prompts
- “/colm-experiments”
What it can do on your machine
Read from SKILL.md and the folder at commit 932eb23. It shows what the files ask for, not the result of running them.
Tool permissions
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Runs code
No scripts in the folder and no shell commands in SKILL.md (its code samples are yaml).
From the folder's file list and the shell code blocks in SKILL.md.
Network
No URLs in SKILL.md.
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Context cost
Colm Experiments loads about 1.6k tokens when it runs. Until then it costs about 82 tokens; SKILL.md has 774 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
Safety
Auto-check passed
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.
SKILL.md
The full file from brycewang-stanford/Awesome-Journal-Skills at commit 932eb23, republished under its MIT licence (© brycewang-stanford). 774 words, ~1,637 tokens.
Download SKILL.mdSave it as
.claude/skills/colm-experiments/SKILL.md (or your agent's skills folder).- name
- colm-experiments
- description
- Use when designing or auditing the empirical core of a COLM paper — contamination analysis for evaluation data, fair baselines under matched prompting and compute, pinned model versions and decoding parameters, uncertainty over runs and samples, scaling coverage, and honest reporting of API-model comparisons.
COLM Experiments
COLM's reviewer pool works on language models daily, which changes what "rigorous" means: the failure modes they hunt are not missing error bars alone but the LM-specific ones — leaked test sets, un-tuned baselines, drifting API models, and decoding settings that quietly decide the result. Build the evaluation so those four attacks fail.
The four LM-specific attacks on your results
1. Contamination. Any public benchmark plausibly overlaps the pre-training data of any recent model. For each evaluation set, either (a) run and report an overlap analysis against known corpora, (b) use dated or freshly generated items and say so, or (c) state explicitly that contamination could not be assessed and bound what it could explain. Silence is the only unacceptable option — and a contamination paragraph placed by the results, not buried in an appendix, defuses the rebuttal question before it is asked.
2. Baseline fairness. "Our method beats the baseline" means nothing if the baseline got one prompt attempt and your method got fifty. Matched budgets are the rule: same prompt-engineering effort, same number of few-shot exemplars, same decoding search, same token budget. Log the effort spent tuning each system; the appendix table of "prompts tried per method" is disarming in review.
3. Version drift. Closed models change behind stable-looking names. Pin every API model with its full version string and query dates; pin every open model with a checkpoint revision hash; pin the evaluation harness commit, because scoring implementations drift too. A result on an unpinned model is a rumor.
4. Decoding sensitivity. Temperature, top-p, max tokens, stop sequences, and the system prompt are experimental variables. Report them per experiment, and when the headline claim is close, show it holds under at least one alternative decoding configuration.
Uncertainty that matches the noise source
LM evaluations have several distinct randomness sources; report the one that actually varies in your setup:
| Noise source | When it applies | Report |
|---|---|---|
| Sampling (temperature > 0) | Any stochastic decoding | n samples per item, mean ± CI |
| Training seed | You trained/fine-tuned the model | ≥ 3 seeds where budget allows; else say why not |
| Prompt phrasing | Any prompted evaluation | Sensitivity over ≥ 3 paraphrases for headline numbers |
| Few-shot exemplar choice | In-context learning | Resampled exemplar sets |
| Data ordering / split | Custom splits | Multiple splits or a fixed public split, justified |
Greedy decoding does not make results deterministic across API replicas or hardware; say what you observed, not what the temperature parameter promises.
One config per number
Every reported number should trace to a frozen config. A reviewer (or you, during the May rebuttal) must be able to regenerate any cell:
# eval/run-042.yaml — one file per reported table cell group
model:
name: <open-model-id>
revision: <checkpoint-hash> # or: api_version + query_date range
tokenizer_revision: <hash>
harness: {repo: <anonymized>, commit: <sha>}
prompt_file: prompts/task-a/v3.txt # verbatim, including system prompt
decoding: {temperature: 0.0, top_p: 1.0, max_tokens: 512, stop: ["\n\n"]}
n_samples_per_item: 5
seeds: [13, 41, 97]
compute_log: logs/run-042.cost # GPU-hours or API spendCompute disclosure
Report the compute for training and evaluation: GPU type and hours for open models, API spend or token counts for closed ones, plus the total including failed runs if it changes the picture materially. COLM's community treats compute as part of the claim — a method that wins by 1 point at 40× evaluation cost is a different result — and disclosure is what makes the follow-up work possible for smaller labs.
Show full SKILL.md (275 more words)Show less
Scaling and coverage
- If you claim a phenomenon "in LMs," show it across ≥ 2 model families and ≥ 2 scales, or scope the claim to the models tested.
- If you claim a method helps, include the strongest simple baseline (better prompting, more samples, self-consistency) — COLM reviewers reflexively ask whether inference-time compute alone reproduces the gain.
- Ablate what your method adds, not what it is built on.
Human and model-judge evaluation
Two further COLM-relevant rigor points, because so many LM claims rest on judged outputs rather than exact-match scoring:
- Model-as-judge is a measurement instrument with its own error. Report the judge model (pinned, like every other model), the judging prompt verbatim, its agreement with human raters on a calibration subset, and known judge biases (position, length, self-preference) that your protocol controls for. Under the 2026 COLM policy, LLM-based evaluation is an explicit disclosure category — the methodological writeup and the policy disclosure should tell the same story.
- Human evaluation needs the same design care as a user study: rater count and recruitment, instructions (verbatim, in the appendix), pay, inter-rater agreement, and item sampling. "Three of the authors rated outputs" is a disclosure, not a protocol — say it plainly if that is what happened.
Design review checklist
- Contamination story exists for every evaluation set (analysis, dated data, or explicit inability statement).
- Budgets matched across systems; tuning effort logged per method.
- Every model, tokenizer, and harness pinned; API query dates recorded.
- Decoding parameters reported per experiment; headline claims checked under one alternative config.
- Uncertainty reported for the actual noise source, with n stated.
- Compute disclosed for training and evaluation.
- Claims scoped to tested models/scales unless coverage justifies more.
Output format
[Evidence verdict] survives the four attacks / vulnerable to: <which>
[Contamination] handled by <analysis / dated data / inability statement> per dataset
[Pinning] models ▢ tokenizers ▢ harness ▢ query-dates ▢
[Uncertainty] <noise source(s)> reported with n = <...>
[Compute] disclosed: <train / eval / both / missing>
[Ordered fixes] <cheapest credibility gain first>© brycewang-stanford, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
Files
Just SKILL.md in COLM-Skills/skills/colm-experiments of brycewang-stanford/Awesome-Journal-Skills.
Open the folder on GitHubat commit 932eb23
Compare with similar skills
Colm Experiments next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Colm Experiments this skillbrycewang-stanford/Awesome-Journal-Skills | 1.2k | — | ~1.6k | Automated safety check: Pass | MIT | |
| Prompt Improverseverity1/claude-code-prompt-improver | 1.9k | 2 repos | ~1.7k | Automated safety check: Pass | MIT | |
| Prompt Engineering Patternsynulihao/AgentSkillOS | 617 | 15 repos | ~1.7k | Automated safety check: Pass | None | |
| Patch CreationPiebald-AI/tweakcc | 2.5k | — | ~1.6k | Automated safety check: Pass | MIT | |
| LLM Application DevMoizIbnYousaf/ai-agent-skills | 1.1k | 2 repos | ~1.3k | Automated safety check: Pass | MIT | |
| Senior Prompt Engineermaslennikov-ig/claude-code-orchestrator-kit | 260 | 4 repos | ~1.4k | Automated safety check: Pass | Custom licence |
Similar skills
Prompt Engineering Patterns
ynulihao/AgentSkillOS
Master advanced prompt engineering techniques to maximize LLM performance, reliability, and controllability in production.
Patch Creation
Piebald-AI/tweakcc
Create and register new patches for tweakcc. An agent skill from Piebald-AI/tweakcc.
LLM Application Dev
MoizIbnYousaf/ai-agent-skills
Building applications with Large Language Models - prompt engineering, RAG patterns, and LLM integration.
Senior Prompt Engineer
maslennikov-ig/claude-code-orchestrator-kit
Provides reference guides and Python scripts for prompt optimization, RAG evaluation, and agent orchestration when building or tuning LLM systems.
Codex Fable5
baskduf/FableCodex
Apply a Claude Fable 5 inspired operating style inside Codex.
More from brycewang-stanford/Awesome-Journal-Skills
All 2,387 skills in this repoAaag Data Analysis
brycewang-stanford/Awesome-Journal-Skills
A skill your agent uses when running and reporting the analysis for an Annals of the American Association of Geographers manuscript — spatial statistics and modeling, remote-sensing accuracy, or…
Aaag Literature Positioning
brycewang-stanford/Awesome-Journal-Skills
A skill your agent uses when positioning an Annals of the American Association of Geographers manuscript in the literature — engaging geographic scholarship across the relevant area and the…
Aaag Rebuttal
brycewang-stanford/Awesome-Journal-Skills
A skill your agent uses when responding to an Annals of the American Association of Geographers decision letter (major/minor revision) — building a point-by-point response to the subject editor and…
Aaag Research Design
brycewang-stanford/Awesome-Journal-Skills
A skill your agent uses when defending the research design of an Annals of the American Association of Geographers manuscript — spatial/quantitative analysis and GIScience, remote-sensing and…
Aaag Review Process
brycewang-stanford/Awesome-Journal-Skills
A skill your agent uses when you need to understand how the Annals of the American Association of Geographers evaluates a manuscript — double-anonymous review routed through a subject editor by…
Aaag Submission
brycewang-stanford/Awesome-Journal-Skills
A skill your agent uses when running the final pre-submission preflight for the Annals of the American Association of Geographers via ScholarOne Manuscripts — area/article-type selection…
Related
Categories
Questions about Colm Experiments
What does Colm Experiments do?
A skill your agent uses when designing or auditing the empirical core of a COLM paper — contamination analysis for evaluation data, fair baselines under matched prompting and compute, pinned model…. Colm Experiments is an agent skill from brycewang-stanford/Awesome-Journal-Skills. Use when designing or auditing the empirical core of a COLM paper — contamination analysis for evaluation data, fair baselines under matched prompting and compute, pinned model versions and decoding parameters, uncertainty over runs and samples, scaling coverage, and honest reporting of API-model comparisons.
When should I use Colm Experiments?
Colm Experiments fits situations like: auditing the empirical core of a COLM paper — contamination analysis for evaluation data; fair baselines under matched prompting and compute; pinned model versions and decoding parameters; uncertainty over runs and samples.
How do I install Colm Experiments in Claude Code?
Run `npx skills add brycewang-stanford/Awesome-Journal-Skills --skill colm-experiments -a claude-code`. Or copy the skill folder (COLM-Skills/skills/colm-experiments in brycewang-stanford/Awesome-Journal-Skills) into .claude/skills/colm-experiments in your project. Claude Code loads it when a task matches its description.
How do I install Colm Experiments in Codex?
Run `npx skills add brycewang-stanford/Awesome-Journal-Skills --skill colm-experiments -a codex`. Or copy the skill folder (COLM-Skills/skills/colm-experiments in brycewang-stanford/Awesome-Journal-Skills) into .agents/skills/colm-experiments in your project. Codex loads it when a task matches its description.
Can I use Colm Experiments in Cursor, Gemini CLI or GitHub Copilot?
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add brycewang-stanford/Awesome-Journal-Skills --skill colm-experiments -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/colm-experiments, .gemini/skills/colm-experiments, .github/skills/colm-experiments and .opencode/skills/colm-experiments in your project.
What does Colm Experiments need to run?
SKILL.md names no scripts, command-line tools or credentials: Colm Experiments is instructions for the agent only.
Does Colm Experiments access the network?
SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.
Is Colm Experiments safe to install?
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.
What licence does Colm Experiments use?
Colm Experiments is published under the MIT licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.
How many tokens does Colm Experiments use?
About 1.6k tokens (SKILL.md is roughly 6.5k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.
What are the alternatives to Colm Experiments?
Skills that share tags, products or a category with Colm Experiments: Prompt Improver (severity1/claude-code-prompt-improver, 1.9k stars), Prompt Engineering Patterns (ynulihao/AgentSkillOS, 617 stars), Patch Creation (Piebald-AI/tweakcc, 2.5k stars) and LLM Application Dev (MoizIbnYousaf/ai-agent-skills, 1.1k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
Who maintains Colm Experiments?
brycewang-stanford (a GitHub user) maintains it in brycewang-stanford/Awesome-Journal-Skills, which has 1,219 GitHub stars. The repository holds 2,387 skills in this directory. The repository was last updated on September 27, 2026.
Source: brycewang-stanford/Awesome-Journal-Skills on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.