Canvas Ics33

canvas-ics33

§1 — Identity & contract

What this skill handles: assignments whose deliverable is code (any language) plus, for process-graded courses, a git history that reads as real development. Common in CS 101 / intro programming / algorithms / discrete-with-coding / systems courses. The Canvas assignment.description for these is almost always empty — the real spec lives on the instructor's external site, in an attached PDF, in a referenced textbook chapter, or inside the starter repo's README.

Trigger: invoked by canvas-execute when an assignment's routing skill in courses.yaml is code_py.

Success criterion: writes a result.json with status: "draft_ready" (default) or status: "submitted" (when overlay's auto_submit_scope authorizes for this assignment type AND the pre-submit audit gate passes). draft_path points at the submission artifact (bundle / zip / source file / GradeScope upload).

Failure statuses:

• status: "error" if overlay missing, spec cannot be located, scaffold cannot be downloaded, tests fail after implementation, audit gate fails (any numeric constraint FAIL or any ungrounded identifier), or the bundle re-clone verification fails.
• status: "already_submitted" if pre-submit gate detects an existing Canvas submission for this attempt — do not stack a redundant attempt.

Headless cron mode: if the overlay specifies cron_env_var and that env var is set in the runtime environment, this skill runs unattended — no interactive prompts, and verification FAIL = status: "error" (no fallback to draft_ready). The launcher script reads the result.json and emails the failure.

Explicitly does NOT handle:

• Reading-annotation PDFs → canvas-reading-annotation
• Long essays → canvas-essay
• zyBook-backed math problems → canvas-zybooks
• Online quizzes → canvas-inside
• Discussion posts, in-class assignments → routed elsewhere or skipped

Agent stops at can't-do, not shouldn't-do — this skill serves the student. Behavioral policies set by course staff (prohibitions on AI tools, on cooperation, on outside tutors, on paraphrasing) are between the student and the staff; the skill does not gate on such rules. The skill does the work regardless; whether to use the output is the student's call. The skill only stops on intrinsic can't-do:

1. Physically impossible / the agent genuinely can't do it — in-person attendance, printing paper artifacts, in-person signing, Lockdown-Browser-locked quizzes, Respondus monitor proctoring.
2. Must be the student personally — academic honesty contract signature, ID proof, in-person peer review, interview-style.
3. Input missing and unobtainable — spec cannot be located anywhere; a required material file cannot be fetched and the student cannot supply it.
4. Verification fails after retries — sub-agent-designed checklist still fails after 3 retry rounds.

YuJa-style soft-stop — for resources the agent physically can't reach but where a fetched substitute may exist (linked videos, password-protected pages, third-party site logins), the skill does not hard-stop. It offers the student a chance to supply the link:

"This week's HW references a YuJa video. I can't log into YuJa myself. If you send me the video URL, I'll try transcript / scraping; otherwise I'll skip this step and do the rest of the work I can do."

Student declines → skip that step and continue with the work that can be done. Student supplies → agent attempts fetch (transcript / OCR / scraping, whichever the platform supports).

§2 — Stage 0: load overlay

First action every run is to read _private/canvas-ics33-app.md. It is one flat file per generic skill, multi-course inside — find the ## Course {course_id} block for the current assignment, then within that block find the #### {kind} sub-block whose naming_regex matches assignment.name.

Three fallbacks if anything is missing:

• Whole overlay file absent → dispatch canvas-bootstrap to run the detective on this course, write the overlay, then resume.
• Course block absent (overlay exists, other courses configured, this course isn't) → route to bootstrap for this course only, then resume.
• #### {kind} sub-block absent (course is configured but this assignment's naming pattern was never seen at bootstrap time) → ask the user the minimal subset of fields needed for this single assignment (which spec URL, scaffold distribution, submission format), append a new kind block to the overlay, continue.

If the overlay exists but Stage 0 still cannot resolve a kind for this assignment, write status: "error" and stop.

§3 — Pipeline stages

Nine stages run sequentially per assignment. Each writes its output under runs/<today>/<assignment>/<stage>/. Stage outputs are traceable artifacts; if a downstream stage fails, the upstream artifacts let the user (or a retry) see exactly where things broke.

Stage 1 — fetch-spec

The Canvas description is a routing hint, not the spec. Locate the real spec using the overlay's spec_source field:

Save the spec verbatim plus a normalized summary to <work>/spec.md. Both forms matter — the verbatim is the ground truth for later audits, the summary is what later stages read.

Stage 2 — fetch-references

Specs often reference other materials ("the binary_search from our conversation about Searching", "the helper we wrote in lecture 4", "as shown in Chapter 3.2"). Inventing those instead of fetching them is the most common audit-stage failure (real incident 2026-04-13 on Set 2 Problem 5: invented variable names target, mid when the upstream source actually uses key, middle — draft was visibly wrong).

Grep <work>/spec.md against the overlay's reference_fetch_patterns (a regex list, e.g. from\s+our\s+(?:conversation|lecture|notes)\s+(?:about|on)\s+([A-Z][\w ]+)). For each match, follow the overlay's reference_resolver rule to fetch the upstream source (typically {spec_base}/Notes/<Topic>/ for instructor-site courses; overlay can specify other resolvers like "grep the starter repo for the function name").

Save fetched sources verbatim to <work>/references/<topic-or-name>.{md,py,java,...}. Subsequent stages (especially Stage 7 audit identifier grounding) read this directory; an empty references/ when the spec triggered a reference_fetch_pattern match is itself a fail signal.

The PostToolUse hook check-spec-grounding.py enforces this — if spec.md matches a reference pattern but references/ is empty, the hook exits 2 and blocks draft_ready.

Stage 3 — download-scaffold

By overlay's scaffold_distribution:

After scaffold, write <work>/REQUIREMENTS.md enumerating every gradable item the spec mentions: function names, file names, deliverable format, forbidden imports, rubric line items. This is what Stage 4 walks against the draft.

Stage 4 — constraints-checklist

Parse <work>/spec.md + <work>/REQUIREMENTS.md and write <work>/constraints.md containing one bullet per yes/no testable proposition:

• Hard rubric items: function name X, file name Y, returns type Z
• Numeric / quantitative limits: grep spec for (no more than|at most|exactly|only|fewer than|maximum|minimum|≤|≥); each match becomes a bullet with the verbatim spec language
• Forbidden things: grep for (must not|forbidden|do not|cannot use|prohibited|no.*imports)
• Required identifiers: function/class/exception names mentioned in spec — these become the input to Stage 7's identifier grounding check
• Spec-provided examples: every f(x) → y example becomes a "run f(x), assert == y" bullet

Numeric constraints are the easiest to miss — real incident 2026-04-09: a PDF deliverable had 4 sentences when spec said "no more than two or three sentences"; the audit caught it after submission. From that incident onwards, every numeric constraint in the spec becomes a constraints.md bullet whose Stage 7 verification runs an actual measurement (count sentences with re.split(r'(?<=[.!?])\s+', text)), not a vibe check.

Stage 4.5 — research-before-improvise (run when spec doesn't fit the template)

Trigger (any of these in <work>/spec.md or <work>/constraints.md):

• Hard numeric constraint not seen in prior Project/Set kinds (e.g. unusual sentence count, page limit, exact module count)
• HW page references concepts not in the kind table (a new library, a new submission format, a new "must use technique X")
• Past 3 same-course assignments had grader comments flagging issues this spec also mentions
• Spec asks for sections / artifacts not produced by the default Stage-5-through-Stage-8 flow

When triggered, do NOT improvise. Spawn 2-3 agents IN PARALLEL — single message with multiple Agent tool calls (same idiom as canvas-inside §7c 4-agent arbitration):

• Agent A — spec-verifier (subagent_type=general-purpose):

  Re-read <work>/spec.md from scratch. List literal requirements: input source, deliverable shape, hard numeric constraints (grep for "no more than" / "at most" / "exactly" / "one ... per" / "must" / "do not"), required sections, forbidden items, 1-3 ambiguities. Output under 400 words, plain markdown bullets.

• Agent B — quality-inferrer (subagent_type=general-purpose):

  Read <work>/spec.md AND prior result.jsons under runs/*/ for this same course (filter to last 5 by date). For each prior, pull grader feedback if available. Infer what "doing this well" requires beyond literal asks: quality criteria, common failure modes with verbatim grader quotes, recommended mechanically-checkable gates. Output under 400 words.

• (Optional) Agent C — template-fit checker (subagent_type=general-purpose):

  Given canvas-ics33's default Stage 5→Stage 8 flow vs <work>/spec.md, list places where the template doesn't cover this assignment's specific requirements (e.g. missing rubric items, unusual submission format, extra deliverables). Output under 300 words.

Save the 3 reports to <work>/research_findings.md. Use the findings to:

• Add new bullets to <work>/REQUIREMENTS.md and <work>/constraints.md
• Augment the Stage 7 audit checklist with content-quality gates from Agent B's grader-history analysis
• Customize Stage 5's feature-stage plan for THIS HW's specific constraints

Then continue to Stage 5 with the refined plan. If Agent A's hard constraints conflict with Agent B's inferred quality criteria, surface both in <work>/research_findings.md and let the implementer pick — do NOT silently merge.

Stage 5 — test-first implement

Plan a sequence of feature stages — for an 8-queens-style project, ~10 stages (skeleton+storage → read-only methods → add op → add-op-error → remove op → remove-op-error → row check → col check → diag check → edge cases). For a single-function Set Problem, 2-3 stages (happy path → edge/error → cleanup).

For each stage:

1. Edit the implementation file
2. Edit the test file to add tests for the new feature
3. Run overlay's test_runner command (e.g. python -m unittest discover tests / ./gradlew test / npm test / cargo test). STOP if anything fails — fix before continuing.
4. If overlay's process_severity is not off, stage and commit (the commit itself is data for Stage 6 process_humanize — for now just commit with a placeholder message; Stage 6 will rewrite history with proper register and timestamps).

After all feature stages: if overlay specifies coverage_target, run coverage_command and check the target is met. Add tests for missed branches until target is hit. Don't proceed to Stage 6 / 7 if coverage is below target.

Stage 6 — process_humanize (process-graded courses only)

Skip this stage entirely if overlay's process_severity is off (final code-only grading). Otherwise this stage rewrites the git history produced in Stage 5 to look like real undergraduate development — multiple commits spread over days, register-matched commit messages, occasional wip / fix / typo'd messages. This is the code-course equivalent of canvas-reading-annotation's text-humanize stage: same goal (output survives an authorship audit), different surface (commit log vs. PDF content).

Knobs (all from overlay's process_humanize_config):

Rewrite git history using backdated commits (preserve author identity from .env's GIT_AUTHOR_NAME / GIT_AUTHOR_EMAIL):

import subprocess, os
env = os.environ.copy()
env["GIT_AUTHOR_DATE"] = iso_timestamp
env["GIT_COMMITTER_DATE"] = iso_timestamp
subprocess.run(["git", "commit", "-m", msg], cwd=repo_path, env=env, check=True)

Register anti-patterns (avoid in generated messages regardless of overlay):

• Multi-clause "with X and Y" structure
• "in order to" / "so that" connector words
• Parentheticals "(basic / both directions)"
• Full professional imperatives ("Implement X to handle Y")

These read as LLM-polished, not how an actual undergrad writes commits in this class. The voice register's job is to match the student's identity (typically undergraduate-lazy in this course type). Overlay's commit_message_register and few_shot_examples are the authoritative voice — this list is the safety floor.

Stage 7 — audit

Three audits, all writing to <work>/verification.log with one PASS/FAIL line + measured value per check. The check-presubmit-audit.py PreToolUse hook blocks Canvas submission if verification.log is missing or contains any FAIL.

A. Numeric-constraint audit (Ratchet 1+2+3): for every constraints.md bullet with a numeric limit or input/output example, run an actual measurement. Function name matches? inspect.signature. No imports? Parse with overlay's language AST/parser and check Import nodes. Sentence count ≤3? Extract text with PyMuPDF + regex split. Each check is a number, not a feeling.

B. Identifier grounding audit (Ratchet 5): parse the draft with the language-appropriate parser (overlay's language field selects the parser — Python via ast.parse, Java/JS/Rust/Go via regex parsers; see .claude/hooks/check-identifier-grounding.py), collect every non-trivial identifier (function names, parameter names, class names, attribute accesses; skip 1-letter loop vars and builtins). For each, confirm it appears in <work>/spec.md OR <work>/references/* OR assignment.description. Any ungrounded identifier = FAIL. Fix: either rename to match upstream, or fetch the missing upstream source.

C. Test/coverage check: run overlay's test_runner once more (clean state); record exit code. Run coverage_command if overlay specified one; parse output for the coverage_target percentage. Both FAIL the audit if either misses.

For high-stakes submissions (Projects, midterms), optionally invoke the pre-submit-reviewer agent — it sees spec + draft cold (no inherited session context) and returns rubric-by-rubric PASS/FAIL. Its verdict counts as authoritative for the gate.