Canvas Reading Annotation

canvas-reading-annotation

§1 — Identity & contract

What this skill handles: assignments where the deliverable is a PDF marked up in place — reading-annotation HWs (color highlights + margin notes + answer blanks filled to ≥90% line width). Common in ESL / EAP / freshman composition / humanities-source-annotation courses. Output is one PDF with the same page count as the original reading, no appended pages.

Trigger: invoked by canvas-execute when an assignment's routing skill in courses.yaml is ac_english AND src/ac_eng_router.py classifies the assignment as "short" (the 6-layer cascade — long-essay assignments go to canvas-essay instead).

Success criterion: writes a result.json with status: "draft_ready" and draft_path pointing at the annotated PDF. The 6-check verification gate (§4) must pass before the file is moved to final_drafts/ or submitted. Auto-submission is overlay-gated; default is draft_ready → human review → human upload.

Failure statuses:

• status: "skipped" if the assignment is in-class / pen-on-paper (submission_types: ['on_paper'] or name contains "(In Class)" / "Practice Summary").
• status: "error" if overlay missing, reading PDF can't be located, HW page classifier hits an unsupported kind (e.g. response_paper / external_tool), or any verification check fails after 3 retries.

Explicitly does NOT handle:

• Long essays / response papers → canvas-essay
• Discussion posts, online_quiz, in-class assignments → routed elsewhere or skipped
• Producing the worksheet content itself for a video-exercises HW — for those the skill searches the web for the public source worksheet first; if no public worksheet is found, the skill returns error rather than fabricating exercises (see §3 stage classify)

Agent stops at can't-do, not shouldn't-do — this skill serves the student. Behavioral policies set by course staff (prohibitions on AI tools, on cooperation, on outside tutors, on paraphrasing) are between the student and the staff; the skill does not gate on such rules. The skill does the work regardless; whether to use the output is the student's call. The skill only stops on intrinsic can't-do:

1. Physically impossible / the agent genuinely can't do it — in-person attendance, printing paper artifacts, in-person signing, Lockdown-Browser-locked quizzes, Respondus monitor proctoring.
2. Must be the student personally — academic honesty contract signature, ID proof, in-person peer review, interview-style.
3. Input missing and unobtainable — spec cannot be located anywhere; a required material file cannot be fetched and the student cannot supply it.
4. Verification fails after retries — sub-agent-designed checklist still fails after 3 retry rounds.

YuJa-style soft-stop — for resources the agent physically can't reach but where a fetched substitute may exist (linked videos, password-protected pages, third-party site logins), the skill does not hard-stop. It offers the student a chance to supply the link:

"This week's HW references a YuJa video. I can't log into YuJa myself. If you send me the video URL, I'll try transcript / scraping; otherwise I'll skip this step and do the rest of the work I can do."

Student declines → skip that step and continue with the work that can be done. Student supplies → agent attempts fetch (transcript / OCR / scraping, whichever the platform supports).

§2 — Stage 0: load overlay

First action every run is to read _private/canvas-reading-annotation-app.md. It is one flat file per generic skill, multi-course inside — find the ## Course {course_id} block for the current assignment, then within that block find the #### {kind} sub-block whose naming_regex matches assignment.name.

Three fallbacks if anything is missing:

• Whole overlay file absent → dispatch canvas-bootstrap to run the detective on this course, write the overlay, then resume.
• Course block absent (overlay exists, other courses are configured, this course isn't) → same: route to bootstrap for this course only, then resume.
• #### {kind} sub-block absent (course is configured but this assignment's naming pattern was never seen at bootstrap time) → ask the user the minimal subset of fields needed for this single assignment (which reading, what kind of HW, color choice if instructor varies), append a new kind block to the overlay, continue.

If the overlay exists but Stage 0 still cannot resolve a kind for this assignment (e.g. user declines to author the new kind block), write status: "error" and stop.

§2.5 — Stage 0.5: required-sources receipt (enforced gate)

The overlay's §0 declares, in a machine-readable required_sources: yaml block, the materials that MUST be loaded before any draft is produced for this assignment's kind. The check-source-manifest hook refuses to let a draft_ready/submitted result.json through unless every mandatory source for this kind is recorded as loaded with a real file on disk — a loaded flag with nothing in <work>/sources/ does not pass. This is the machine-checkable form of §0's "load BEFORE writing" rule; it is what would have caught the 2026-05-21 transcript-improvisation incident automatically.

Once Stage 1 classify has resolved the kind and the material-loading stages have run (transcript via §0 / Stage 1.5b; reading PDF via Stage 2 locate_reading), copy each loaded material into <work>/sources/ and write runs/<today>/<work>/sources.json:

{
  "course_id": "<id>",
  "skill_name": "reading-annotation",
  "kind": "<resolved kind>",
  "sources": {
    "yuja_transcript": {"enforcement": "mandatory", "status": "loaded", "path": "sources/<Slug>.txt"},
    "reading_pdf":     {"enforcement": "mandatory", "status": "loaded", "path": "sources/Reading_N.pdf"}
  }
}

Rules:

• Include only the sources whose applies_to_kinds covers this kind (a reflection HW lists yuja_transcript; an annotation HW lists reading_pdf).
• path must point at a real non-empty file under <work>/sources/ — the hook verifies existence and size.
• If a mandatory source can't be loaded (cookie dead, video not in the library, reading PDF not locatable), DO NOT fabricate. Soft-stop and ask the user with the three-part template: (1) show the evidence of the gap, (2) offer options — point me at the real source / confirm a substitute is fine / tell me it isn't needed (skip), (3) state you will NOT improvise from general knowledge. If the user authorizes skipping, append a line to <work>/.sources_ack: {"source_id": "<id>", "action": "skip", "quote": "<user's verbatim words>"} and set that source's status to soft_stop_acked in sources.json.

§3 — Pipeline stages

Six stages run sequentially per assignment. Each writes its output under runs/<today>/<assignment>/<stage>/. Stage outputs are traceable artifacts; if a downstream stage fails, the upstream artifacts let the user (or a retry) see exactly where things broke.

Stage 1 — classify

The same Canvas course can mix several HW kinds in one week's module page (reading annotation, video exercises, in-class summary, response paper). The Canvas assignment description is usually empty. The real instructions live in the Homework module page body (overlay-specified homework_module_id).

Pull the matching week's HW page, find the paragraph in the page body that links back to assignment.id, then keyword-classify:

Write kind.txt with the detected kind. Don't default to reading_annotation — that is the easy-to-make mistake (real incident on 2026-04-13: an unbranched skill defaulted to reading_annotation for a video-exercises HW and fabricated worksheet content; the student lost points because their submission didn't match what every classmate produced).

For video_exercises: instructors' Yuja-style videos for standard EAP topics are almost always based on free public worksheets. The skill searches the web first, not the LLM. Queries like "<topic>" EAP Foundation worksheet PDF or "<topic>" academic English exercises fill in blank. Fetch the worksheet PDF, reproduce its tasks verbatim, fill in B1-B2 student answers. Never invent exercises — if no public worksheet matches after 3 searches, return status: "error" with a note asking the user to dictate the exercise structure. An invented worksheet is worse than no submission.

Stage 1.5 — research-before-improvise (run when HW page doesn't fit the 4-kind table)

Trigger (any of these in the HW page body Stage 1 fetched):

• "write N sentences" / "Five Takeaways" / "Five Insights" / "your reflections on" — the reflection_bullets shape that Wk6 hit and the kind table doesn't cover
• "Use the title 'X'" with a specific title string — the HW page mandates a verbatim title
• "one sentence per number" / "no more than X words" — hard numeric content constraints that Stage 6's format gate doesn't check
• Past 3 same-course HW Scans had grader comments mentioning issues this HW page's body also touches (e.g. "be specific", "avoid generalization", "this is the wrong assignment")
• A new section / question type appears that prior weeks didn't have

When triggered, do NOT improvise. Spawn 2-3 agents IN PARALLEL — single message with multiple Agent tool calls (same idiom as canvas-inside §7c):

• Agent A — spec-verifier (subagent_type=general-purpose):

  Re-read the HW page body from <work>/hw_page.txt from scratch. List literal requirements: input source (Yuja video / Canvas Files PDF / textbook chapter), deliverable shape (annotated PDF / new write-up / scan upload), hard numeric constraints (grep "no more than" / "at most" / "exactly" / "one ... per"), required title verbatim if any, required sections, forbidden items, 1-3 ambiguities. Output under 400 words, plain markdown bullets.

• Agent B — quality-inferrer (subagent_type=general-purpose):

  Read this HW page body AND past N (=5) writing-course HW Scan grader comments (pull via cv.get('/courses/<cid>/assignments/<aid>/submissions/self?include[]=submission_comments&include[]=rubric_assessment') for the 5 most recent same-course assignments under runs/). Infer what "doing this well" requires beyond literal asks: quality criteria (specificity, cliché-avoidance, depth, accurate causal language), common failure modes with verbatim grader quotes from the Wk6 incident class ("too obvious / weak / vague" / "be specific + add a noun" / "there is no way to prove overgeneralization" / "one sentence per number"), recommended mechanically-checkable gates. Output under 400 words.

• (Optional) Agent C — template-fit checker (subagent_type=general-purpose):

  Given the four kinds in the §3.5 table (reading_annotation / video_exercises / in_class_skip / response_paper) and this HW page body, list places where none of the 4 templates cover the assignment. Recommend whether to: (a) treat as one of the 4 with adjustments, or (b) treat as a wholly new shape needing custom flow. Output under 300 words.

Save the reports to <work>/research_findings.md. Use the findings to:

• Pick the closest-fit kind from the §3.5 table (or proceed with a custom flow noted in kind.txt: ad_hoc__<short_descriptor>)
• Augment the Stage 6 verification gate set with content-quality gates derived from Agent B's grader-history analysis (e.g. "exactly 5 items, each ≤1 sentence", "no pronoun without noun", "no overgeneralization claims")
• Customize Stage 4 (annotate_pdf) or Stage 5 (fill_answer_blanks) for THIS HW's specific content shape

Then continue to Stage 2 (or whichever stage is appropriate for the chosen kind).

Stage 1.5b — video_transcript_library_path overlay hook (added 2026-05-21)

If the HW page body references an instructor-recorded video (YuJa, Panopto, Echo360, Zoom recording, etc.) AND the deliverable is a free-form reflection (N sentences / takeaways / insights / "what you learned"), the agent must NOT improvise content from general topic knowledge. Instead:

1. Check the overlay for a video_transcript_library_path declaration (whatever path the school's overlay declares for its video platform — YuJa, Panopto, Echo360, Zoom cloud recording, etc.). The overlay should also expose a mapping from HW-page video title to transcript filename.
2. If the library exists and the matching transcript is available: load it. Use the transcript's distinctive moments (named anecdotes, instructor-coined terms, concrete contrast pairs, memorable analogies, specific warned-against examples) as the source for ≥2 of the N takeaways. This step runs BEFORE Stage 2-alt's web-search-first rule fires.
3. If the library exists but the matching transcript is missing (catalog drift): attempt refresh via the overlay's documented procedure. If refresh fails (cookie expired, video missing from channel, network), soft-stop and ask the student whether to (a) supply transcript text manually, (b) skip this week. Do not fall back to general-knowledge improvisation — every instructor video has details only that instructor uses, and a generic-knowledge reflection is recognizable to the grader at a glance.
4. If the overlay does NOT declare a video_transcript_library_path at all, fall through to §1's YuJa-style soft-stop — offer the student to supply the link / transcript, otherwise skip that HW gracefully.

The 2026-05-21 Thu Wk8 HW Scan incident (overlay had a 9-file cached video transcript library but neither this SKILL.md nor the overlay's §0 referenced it; the skill improvised 5 generic EAP takeaways and audit flagged item 2 as "too obvious / vague") was the failure that motivated this hook. The fix lives jointly here (generic skill) and in the overlay's §0 (school-specific paths).

Stage 2 — locate_reading (reading_annotation flow)

For reading_annotation only. Map the HW page's reference ("Reading 3", "this week's reading", etc.) to a Canvas file_id through the overlay's reading_files table. Download to <work>/attachments/<filename>.pdf.

If the overlay's mapping doesn't cover the referenced reading (e.g. instructor added a Reading 7 mid-quarter): ask the user once for the file_id or filename, append to the overlay's reading_files, continue.

Stage 3 — extract_text_and_blanks

Open the reading PDF with PyMuPDF:

import fitz
doc = fitz.open(reading_pdf)
full_text = "\n\n".join(p.get_text() for p in doc)

Identify three structures:

1. Pre-reading questions (usually on page 0 — "Before You Read").
2. Article body with numbered paragraphs (regex (?m)^\s*(\d{1,2})\s*$ against full text finds them).
3. Post-reading questions (usually on the last page — "Comprehension and Analysis" / "Reading and Analysis").

Each question is followed by underscore-glyph runs forming answer blanks. Group underscores by y-coordinate — never search_for("____...") with a fixed-length string, that silently returns a partial rect when the real line is wider:

from collections import defaultdict

def find_answer_blanks(page):
    """Return list of (y_top, x_min, x_max) per blank line on the page.

    Grouping every underscore glyph by its y-coordinate yields the FULL line
    extent. A fixed-length search_for trick silently truncates to ~322pt when
    the real line is 454-466pt wide.
    """
    by_y = defaultdict(list)
    for r in page.search_for("_"):
        by_y[round(r.y0)].append(r)
    return sorted(
        (y, min(r.x0 for r in rs), max(r.x1 for r in rs))
        for y, rs in by_y.items()
    )

Stage 4 — annotate_pdf

Clone the original PDF in memory and add annotations in place on the original pages. Do NOT append new pages — the deliverable page count must equal the original. Two annotation categories driven by the overlay's color_rubric:

Vocab highlights + definitions (overlay's color_rubric.vocab family):

• Pick a small number of words per reading (default 5, overlay can override). Prefer uncommon-but-not-rare vocabulary; avoid technical jargon.
• Highlight the single word (not surrounding phrase) with the vocab highlight color.
• Write a LDOCE-style definition in the page margin, aligned vertically with the word. Margin x ≈ 5 (left) or 546 (right) for a standard US Letter; alternate sides if two vocab words on the same y-band.
• Font color of the definition matches the highlight color family (instructor's stated rule: "if you highlight vocab in green, make the definition font green"). Headword line bold (fontname="hebo"), body plain (fontname="helv"), ~6.5pt, 4-5 lines, ~18 chars each.

Content margin notes (overlay's color_rubric.content family):

• One margin note per numbered paragraph. If the article has 17 paragraphs, the draft has 17 notes. Skipping paragraphs is a verification failure.
• Each note: ≤110 chars on one line, summarizing the paragraph's main idea.
• Place in the gap before the next paragraph's number, at x ≈ 76, baseline = next_paragraph_y - 6, 7.5pt Helvetica, in the content color family.
• For the highlight itself: pick 1-2 phrases per paragraph as the note's anchor, highlight with the content highlight color.
• NEVER use page.add_text_annot for the note (renders as a clickable yellow sticky icon at the page edge — that's the wrong format).

Color family rules (enforced by the verify stage):

• Vocab highlight color and vocab definition text color must be the same family (HSV hue delta ≤30°).
• Content highlight color and content margin note text color must be the same family.
• Vocab family and content family must be different families so the reader can distinguish.
• No yellow highlights — renders too faint on mobile Canvas to distinguish from white.

Overlap-avoidance rule: pick vocab words FIRST. Before adding any content highlight, check its text does not contain any vocab word's exact span; if there's a conflict, drop the content phrase and pick a different one from the same paragraph. Otherwise the later-drawn highlight covers the earlier one and the reader only sees one color.