Research Proposal

Research Proposal Loop

The artifact is a research proposal; the feedback signal is ScholarEval (literature-grounded Soundness + Contribution) turned into a 0-100 grade by a Judge against the fixed rubric.md. Each iteration evaluates → grades → revises until the grade clears <pass_threshold> or the budget runs out.

North star. Clearing the threshold is the stopping condition, not the goal. The goal is the strongest, most novel, genuinely publishable version the proposal can honestly become — every revision should ask "does this make the work more significant and more novel?", not just "does this patch a flaw?". This is grounded ambition: the lift comes from better-justified methods, a sharper-but-defensible novelty claim, and stronger baselines, all backed by real retrieved evidence. Overclaiming lowers the grade (evidence gate + Contribution axis); it never raises it.

The cast (all in this folder):

• roles/ScholarEval.md — the two-module literature-grounded evaluator; emits scholareval.json.
• roles/Judge.md — grades the feedback → 0-100 + ranked fixes; emits verdict.json (decides pass).
• roles/Reviser.md — rewrites the proposal to address the fixes (guards the research intent).
• rubrics/rubric.md — the fixed grading rubric (the Judge never edits it).
• schemas/scholareval.schema.json, schemas/verdict.schema.json — the two validated outputs.

Spawn-or-degrade. On Claude Code, spawn ScholarEval / Reviser as real Agent subagents; otherwise adopt each role inline. You are the orchestrator and the Judge.

When to use

Use when a written proposal exists and the user wants it pushed past a quality bar with literature-grounded critique. Default: run the full evaluate→grade→revise loop below. Escape hatch: if the user only wants the critique (no rewriting), run one ScholarEval + Judge pass and stop. Not for writing a proposal from a blank page, and not for a standalone literature survey.

Setup

Resolve bindings interactively. If loop.run.yaml exists in the working dir, load it and skip to the loop. Otherwise: on Claude Code (the AskUserQuestion tool is available) infer a likely value for each binding and present it as the recommended option; on other hosts ask each as a quoted plain-text prompt. Then write loop.run.yaml (format: examples/run.example.yaml) and confirm every value, <intent>, and the live/degraded literature tier before creating any other files.

Evaluation depth dial (<eval_scale> caps per iteration — methods · dimensions examined · queries each · papers read full-text):

Literature toolchain. Paper search goes through the sibling literature-search skill — resolve <lit_skill_dir> (it installs as a sibling, e.g. ~/.claude/skills/literature-search/), <lit_py> = python3, and <lit> = <lit_skill_dir>/tools/lit_search.py; append --cache-dir <sandbox_root>/literature/.cache after a subcommand to reuse the cache. Confirm <lit> --help works at setup; if the skill is absent, tell the user and either install it (copy the repo's loops/literature-search folder into ~/.claude/skills/) or degrade all retrieval to WebSearch/WebFetch (no ranked snippets or citation-graph expansion). The keyless S2 + arXiv core (search/snippet/cite/fulltext) needs no setup; a free S2_API_KEY makes snippet+cite (ScholarEval's two defining moves) reliable.

API key (optional, never block). The literature-search skill owns the key convention: run <lit> keys --init, then have the user fill the printed keys.env themselves and never paste secrets into chat. Re-run <lit> keys to confirm presence and record the tier in loop.run.yaml (literature_tiers, presence only). A missing key just degrades to the keyless pool → WebSearch.

Initialise the sandbox once bindings are confirmed:

<sandbox_root>/
├── loop.run.yaml        ← resolved bindings + <intent> + grade_weights + literature_tiers
├── ledger.tsv           ← header only (see Ledger)
├── literature/.cache/   ← lit_search on-disk cache
└── iter1/proposal.md    ← the input proposal (the baseline)