Agent skill

Asplos Artifact Evaluation

by brycewang-stanford in brycewang-stanford/Awesome-Journal-Skills

A skill your agent uses when preparing an ASPLOS artifact for the post-acceptance evaluation committee — writing the ae.tex Artifact Appendix with software/hardware/dataset dependencies, targeting…

MITAuto-check passedDocuments & Office

Install Asplos Artifact Evaluation

skills CLI
$ npx skills add brycewang-stanford/Awesome-Journal-Skills --skill asplos-artifact-evaluation -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install brycewang-stanford/Awesome-Journal-Skills asplos-artifact-evaluation --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/brycewang-stanford/Awesome-Journal-Skills.git skills-src && mkdir -p .claude/skills && cp -r skills-src/ASPLOS-Skills/skills/asplos-artifact-evaluation .claude/skills/asplos-artifact-evaluation && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
asplos-artifact-evaluation
GitHub stars
1.2k
Token cost
~1.7k tokens
SKILL.md length
763 words
Files
1
Skills in repo
2,387
Repo updated
First seen
Licence
MIT

At a glance

A skill your agent uses when preparing an ASPLOS artifact for the post-acceptance evaluation committee — writing the ae.tex Artifact Appendix with software/hardware/dataset dependencies, targeting…

  • Works in 3 steps: Provide remote access to the platform… → Ship the simulator-backed subset as the… → Offer a scaled-down proxy (smaller FPGA,…
  • Targeting the Available / Functional / Reproducible badges
  • SKILL.md covers The three badges and what each…, The Artifact Appendix is the…, Package layout that evaluators… and Hardware-dependent claims:…, plus 5 more sections
  • Instructions only: no scripts, shell commands, URLs or credentials in SKILL.md

What it does

Asplos Artifact Evaluation is an agent skill from brycewang-stanford/Awesome-Journal-Skills. Use when preparing an ASPLOS artifact for the post-acceptance evaluation committee — writing the ae.tex Artifact Appendix with software/hardware/dataset dependencies, targeting the Available / Functional / Reproducible badges, archiving on a public repository, and planning the collaborative back-and-forth with evaluators.

Its SKILL.md is about 1.7k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.

It sits in Documents & Office, covering LaTeX. The repository describes itself as: Journal-specific Claude Code/Codex skill packs covering mainstream journals — AER, QJE, Nature, Cell, 管理世界, 经济研究 & 200+ more — your fast track to getting published. | 覆盖主流期刊的… The licence is MIT.

When your agent uses it

  • Targeting the Available / Functional / Reproducible badges
  • Archiving on a public repository
  • Planning the collaborative back-and-forth with evaluators

Example prompts

  • “/asplos-artifact-evaluation”

Workflow steps

3 steps, taken from the first numbered list in SKILL.md.

  1. Provide remote access to the platform for the evaluation window (with an
  2. Ship the simulator-backed subset as the reproducible core, and mark the
  3. Offer a scaled-down proxy (smaller FPGA, reduced workload) with an explicit

What it can do on your machine

Read from SKILL.md and the folder at commit 932eb23. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    No scripts in the folder and no shell commands in SKILL.md.

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    No URLs in SKILL.md.

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Asplos Artifact Evaluation loads about 1.7k tokens when it runs. Until then it costs about 88 tokens; SKILL.md has 763 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~88
When it runs · the whole SKILL.md, loaded when a task matches
~1.7k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from brycewang-stanford/Awesome-Journal-Skills at commit 932eb23, republished under its MIT licence (© brycewang-stanford). 763 words, ~1,713 tokens.

Download SKILL.mdSave it as .claude/skills/asplos-artifact-evaluation/SKILL.md (or your agent's skills folder).
name
asplos-artifact-evaluation
description
Use when preparing an ASPLOS artifact for the post-acceptance evaluation committee — writing the ae.tex Artifact Appendix with software/hardware/dataset dependencies, targeting the Available / Functional / Reproducible badges, archiving on a public repository, and planning the collaborative back-and-forth with evaluators.

ASPLOS Artifact Evaluation

Artifact evaluation at ASPLOS is a post-acceptance, opt-in, collaborative process: an independent committee works with authors to validate the paper's key results, and successful artifacts carry badges on the published paper (AE pages, checked 2026-07-08). It is also a tradition the venue itself highlights — systems readers increasingly treat an unbadged systems paper as a weaker citation. Treat AE as part of the publication, budgeted like a small sixth section.

The three badges and what each actually demands

Badge2027 criterion (paraphrased from the AE pages)Practical bar
AvailableArtifact placed on a publicly accessible archival repositoryA DOI-issuing archive (institutional or Zenodo-class); a GitHub URL alone is not archival
FunctionalEvaluators can prepare and run the artifact; they document the steps they followedClean-machine install + a smoke experiment that completes in minutes, not hours
ReproducibleEvaluators validate the paper's key resultsPer-claim run scripts whose output maps visibly onto specific figures/tables

Evaluators assign scores per requested badge and record what they could and could not reproduce — so the artifact's job is to make their success path short and their failure modes diagnosable.

The Artifact Appendix is the contract

ASPLOS 2027 expects an Artifact Appendix built from the provided ae.tex template (or equivalent sections) covering: all software, hardware, and dataset dependencies; the key results to be reproduced; and how to prepare, run, and validate the experiments. Write it as if the evaluator is competent, busy, and using different hardware than yours:

  • Dependencies include the awkward ones: kernel versions, privileged access, BIOS settings, board models, expander firmware — everything from the state ledger in asplos-reproducibility.
  • "Key results" means a selected subset: pick the claims that define the paper, not all 40 bars of every figure. Ambition here creates failure reports.
  • Validation must be decidable: state the expected output and the tolerance within which the claim holds ("ordering preserved; absolute times ±15%").

Package layout that evaluators can navigate blind

text
artifact/
  README.md            # 10-minute quick start + full map
  APPENDIX.pdf         # the ae.tex appendix as submitted
  env/                 # container/VM recipe OR exact install script
  hardware.md          # tiered requirements + what to do without them
  run/
    smoke.sh           # minutes-scale end-to-end sanity check
    claim1_fig6.sh     # one script per key result, named for its figure
    claim2_tab3.sh
  expected/            # reference outputs + tolerance statement per claim
  data/ or data.md     # datasets, or archival pointers + checksums

Hardware-dependent claims: give evaluators a path

The recurring ASPLOS AE failure is a paper whose headline number needs silicon the committee lacks. Acceptable mitigations, in descending order of strength:

  1. Provide remote access to the platform for the evaluation window (with an anonymity-safe access route if the process requires it).
  2. Ship the simulator-backed subset as the reproducible core, and mark the silicon results as demonstrably-run (logs + analysis pipeline included).
  3. Offer a scaled-down proxy (smaller FPGA, reduced workload) with an explicit argument for why the proxy's behavior transfers.

Say which mitigation applies in the appendix, per claim — evaluators score against what you requested, so calibrated requests outperform hopeful ones.

Show full SKILL.md (335 more words)Show less

Collaboration protocol

  • Expect rounds: evaluators report blockers, authors fix and respond. Reserve maintainer time in the weeks after camera-ready (exact 2027 AE dates: 待核实 — confirm at notification).
  • Fix-forward, do not re-argue: an evaluator's confusion is a defect in the README.
  • Keep the artifact frozen at a tagged version during evaluation; hotfixes go on a branch the evaluators are told about.

Common evaluator blockers, pre-empted

Field experience across systems AE committees converges on a short list of first-hour failures, all preventable:

  • Undeclared credentials or licenses — a workload, simulator model, or dataset that needs a registration the evaluator lacks; declare it in the appendix and provide an alternative path.
  • Hidden network assumptions — builds that fetch from internal mirrors or rate-limited hosts; vendor the dependencies or provide the container image.
  • Root-only steps without warning — kernel-module or BIOS-adjacent steps must be flagged up front so the evaluator can pick a sacrificial machine.
  • Hour-scale first feedback — if the smoke test takes an evening, the first blocker report costs a full round trip; minutes-scale smoke tests keep the collaboration inside the calendar.
  • Output the evaluator must interpret — raw logs with no comparator; every claim script should end by printing PASS/FAIL against the tolerance.

Anonymity boundary

AE runs after acceptance, so evaluator-facing materials need not be anonymous — but any artifact pointer placed in the submission itself (an appendix teaser, a footnoted repository) falls under the double-blind rules and must be anonymized end to end: repository owner, commit author strings, container registry paths, and dataset hosting all leak identity. The clean pattern is to keep the submission's artifact story descriptive ("we will submit an artifact covering claims 1-3") and materialize the links only in the Artifact Appendix after notification.

Dry-run protocol

Before submission to the AEC, have a colleague who did not build the artifact execute the README on a clean machine, timing each stage and noting every question they had to ask. Their questions are defects; fix the README, not the colleague. Two such passes typically halve the evaluation rounds.

Output format

text
[Badges requested] available / functional / reproducible — with rationale
[Appendix status] dependencies / key results / prepare-run-validate all drafted: Y/N
[Smoke test] clean-environment runtime: N min · passes: Y/N
[Claim scripts] one per key result, mapped to figure/table: list
[Hardware path] per silicon-dependent claim: access / sim-subset / proxy
[Archive] DOI-issuing repository chosen + deposit dry-run done: Y/N

© brycewang-stanford, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in ASPLOS-Skills/skills/asplos-artifact-evaluation of brycewang-stanford/Awesome-Journal-Skills.

Open the folder on GitHubat commit 932eb23

Compare with similar skills

Asplos Artifact Evaluation next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Asplos Artifact Evaluation compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Asplos Artifact Evaluation this skillbrycewang-stanford/Awesome-Journal-Skills1.2k—~1.7kAutomated safety check: PassMIT
Research Writingalfonso0512/research-writing-skill4901 repos~818Automated safety check: PassMIT
Paper WritingMLNLP-World/Paper-Writing-Tips4.7k—~630Automated safety check: PassNone
Evomath TaoEvoScientist/EvoSkills4782 repos~3.8kAutomated safety check: PassApache-2.0
Math Modeling to EI Conference Paperjihe520/MathModelAgent6.2k—~688Automated safety check: PassNone
PaperjurySpark-To-Paper-Skills/paperjury1.2k—~5.3kAutomated safety check: PassMIT

Similar skills

  • Research Writing

    alfonso0512/research-writing-skill

    科研论文写作助手,提供 30 个 Prompt 模板覆盖论文写作全流程. An agent skill from alfonso0512/research-writing-skill.

    490 GitHub starsUsed in 1 repo~818 tokens
    Documents & OfficeAuto-check passed
  • Paper Writing

    MLNLP-World/Paper-Writing-Tips

    学术论文写作检查与优化助手。基于 MLNLP-World 社区整理的论文写作技巧,帮助检查和优化学术论文。Use when: (1) 检查论文 LaTeX 格式和排版, (2) 优化公式符号使用, (3) 改进图表设计, (4) 润色英文学术表达, (5) 检查参考文献格式, (6) 投稿前终稿检查, (7) 用户询问论文写作技巧或规范。

    4.7k GitHub stars~630 tokensUpdated 14 days ago
    Documents & OfficeAuto-check passed
  • Evomath Tao

    EvoScientist/EvoSkills

    A skill your agent uses whenever the user submits a non-trivial mathematical claim that needs a rigorous proof or audit.

    478 GitHub starsUsed in 2 repos~3.8k tokens
    Documents & OfficeAuto-check passed
  • Rewrites a math modeling competition paper into an English EI conference submission, using a bundled IEEE LaTeX template, with evidence tracked back to the original paper.

    6.2k GitHub stars~688 tokensUpdated 7 days ago
    Documents & OfficeAuto-check passed
  • Paperjury

    Spark-To-Paper-Skills/paperjury

    Three modes for CS-conference papers (CVPR/ICCV/ECCV vision, ACL/EMNLP/NAACL NLP, ICLR/NeurIPS/ICML/AAAI ML).

    1.2k GitHub stars~5.3k tokensUpdated 1 mo ago
    Documents & OfficeAuto-check passed
  • Compiledeck

    scunning1975/MixtapeTools

    Create and compile beautiful Beamer presentations following the Rhetoric of Decks philosophy.

    474 GitHub starsUsed in 2 repos~3.1k tokens
    Documents & OfficeAuto-check passed

More from brycewang-stanford/Awesome-Journal-Skills

All 2,387 skills in this repo
  • Aaag Data Analysis

    brycewang-stanford/Awesome-Journal-Skills

    A skill your agent uses when running and reporting the analysis for an Annals of the American Association of Geographers manuscript — spatial statistics and modeling, remote-sensing accuracy, or…

    1.2k GitHub stars~1.3k tokensUpdated 13 days ago
    Auto-check passed
  • Aaag Literature Positioning

    brycewang-stanford/Awesome-Journal-Skills

    A skill your agent uses when positioning an Annals of the American Association of Geographers manuscript in the literature — engaging geographic scholarship across the relevant area and the…

    1.2k GitHub stars~1.3k tokensUpdated 13 days ago
    Auto-check passed
  • Aaag Rebuttal

    brycewang-stanford/Awesome-Journal-Skills

    A skill your agent uses when responding to an Annals of the American Association of Geographers decision letter (major/minor revision) — building a point-by-point response to the subject editor and…

    1.2k GitHub stars~1.4k tokensUpdated 13 days ago
    Auto-check passed
  • Aaag Research Design

    brycewang-stanford/Awesome-Journal-Skills

    A skill your agent uses when defending the research design of an Annals of the American Association of Geographers manuscript — spatial/quantitative analysis and GIScience, remote-sensing and…

    1.2k GitHub stars~1.4k tokensUpdated 13 days ago
    Auto-check passed
  • Aaag Review Process

    brycewang-stanford/Awesome-Journal-Skills

    A skill your agent uses when you need to understand how the Annals of the American Association of Geographers evaluates a manuscript — double-anonymous review routed through a subject editor by…

    1.2k GitHub stars~1.3k tokensUpdated 13 days ago
    Auto-check passed
  • Aaag Submission

    brycewang-stanford/Awesome-Journal-Skills

    A skill your agent uses when running the final pre-submission preflight for the Annals of the American Association of Geographers via ScholarOne Manuscripts — area/article-type selection…

    1.2k GitHub stars~1.6k tokensUpdated 13 days ago
    Auto-check passed

Questions about Asplos Artifact Evaluation

What does Asplos Artifact Evaluation do?

A skill your agent uses when preparing an ASPLOS artifact for the post-acceptance evaluation committee — writing the ae.tex Artifact Appendix with software/hardware/dataset dependencies, targeting…. Asplos Artifact Evaluation is an agent skill from brycewang-stanford/Awesome-Journal-Skills.tex Artifact Appendix with software/hardware/dataset dependencies, targeting the Available / Functional / Reproducible badges, archiving on a public repository, and planning the collaborative back-and-forth with evaluators.

When should I use Asplos Artifact Evaluation?

Asplos Artifact Evaluation fits situations like: targeting the Available / Functional / Reproducible badges; archiving on a public repository; planning the collaborative back-and-forth with evaluators.

How do I install Asplos Artifact Evaluation in Claude Code?

Run `npx skills add brycewang-stanford/Awesome-Journal-Skills --skill asplos-artifact-evaluation -a claude-code`. Or copy the skill folder (ASPLOS-Skills/skills/asplos-artifact-evaluation in brycewang-stanford/Awesome-Journal-Skills) into .claude/skills/asplos-artifact-evaluation in your project. Claude Code loads it when a task matches its description.

How do I install Asplos Artifact Evaluation in Codex?

Run `npx skills add brycewang-stanford/Awesome-Journal-Skills --skill asplos-artifact-evaluation -a codex`. Or copy the skill folder (ASPLOS-Skills/skills/asplos-artifact-evaluation in brycewang-stanford/Awesome-Journal-Skills) into .agents/skills/asplos-artifact-evaluation in your project. Codex loads it when a task matches its description.

Can I use Asplos Artifact Evaluation in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add brycewang-stanford/Awesome-Journal-Skills --skill asplos-artifact-evaluation -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/asplos-artifact-evaluation, .gemini/skills/asplos-artifact-evaluation, .github/skills/asplos-artifact-evaluation and .opencode/skills/asplos-artifact-evaluation in your project.

What does Asplos Artifact Evaluation need to run?

SKILL.md names no scripts, command-line tools or credentials: Asplos Artifact Evaluation is instructions for the agent only.

Does Asplos Artifact Evaluation access the network?

SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.

Is Asplos Artifact Evaluation safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Asplos Artifact Evaluation use?

Asplos Artifact Evaluation is published under the MIT licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Asplos Artifact Evaluation use?

About 1.7k tokens (SKILL.md is roughly 6.9k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Asplos Artifact Evaluation?

Skills that share tags, products or a category with Asplos Artifact Evaluation: Research Writing (alfonso0512/research-writing-skill, 490 stars), Paper Writing (MLNLP-World/Paper-Writing-Tips, 4.7k stars), Evomath Tao (EvoScientist/EvoSkills, 478 stars) and Math Modeling to EI Conference Paper (jihe520/MathModelAgent, 6.2k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Asplos Artifact Evaluation?

brycewang-stanford (a GitHub user) maintains it in brycewang-stanford/Awesome-Journal-Skills, which has 1,231 GitHub stars. The repository holds 2,387 skills in this directory. The repository was last updated on September 27, 2026.

Source: brycewang-stanford/Awesome-Journal-Skills on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.