Canvas Inside

Canvas Classic Quiz workflow

Handle online_quiz assignments with a real Classic Quiz quiz_id. Supported question types are multiple choice, multiple answers, true/false, matching, short answer, fill-in-multiple-blanks, multiple dropdowns, numerical, and essay. New Quizzes (external_tool with no quiz_id) and online text entries are outside this skill.

Plan approval permits the approved local work only. It never grants authority to start an attempt, save an answer or event, complete an attempt, or retake.

Entry and artifact contract

Require run_dir, course_id, assignment_id, the validated assignment snapshot and approved plan item. Create the only work directory with:

from src.course_artifacts import ensure_stable_work_dir

work_dir = ensure_stable_work_dir(run_dir, course_id, assignment_id)

Its name is exactly course-<course_id>__assignment-<assignment_id>. Keep all quiz evidence there:

quiz_meta.json
readings/
references/
study_notes.md
submission.json
questions.json
questions_simplified.json
agent_passes/
final_answers.json
answer_log.json
attempt-1/
attempt-2/
audit/learning_log.json
result.json

Read _private/canvas-inside-app.md first and select the exact course block. It may contain the course whitelist, recurring quiz scope, instructor framework primer, expected canonical knowledge, human-hours window, rate limit, target score band and retake threshold. If the block is absent, write error with reason_code=missing_course_overlay, recommend a separate canvas-bootstrap run, and stop. Never copy private overlay values into this tracked skill.

1. Classify without starting an attempt

Read the assignment, current submission and quiz metadata. Save the complete quiz object atomically as quiz_meta.json, adding exact course_id, quiz_id and assignment_id; its id must also equal quiz_id. Before answer or complete mutations, Stage 4 adds the current session_id, positive integer attempt, and authorization_receipt_id. Never copy a signature or validation token into this metadata file.

Classify as follows:

Do not infer the quiz shape from its name. If an inaccessible video or third-party source may have a transcript, ask the student for the link. Continue without it only when the remaining authoritative sources are sufficient.

2. Discover the real readings

Do this read-only stage before checking scheduling toggles or mutation authority. It must produce study_notes.md, so a later draft_ready result has a real deliverable.

Write verification.log at the same stage with measured PASS/FAIL lines for metadata capture, required-reading coverage, source traceability, and unresolved placeholders. An unclassified quiz that returns quiz_meta.json must still record a real metadata-capture PASS. No draft_ready result is valid while this log is missing, empty, or contains FAIL.

Use a four-layer source hunt; the Canvas description is only a routing hint.

1. Section/week module: list modules, identify the relevant section or week, then read every Page, download every File, and follow every External URL.
2. Course files plus syllabus: list course files, inspect the syllabus or schedule mapping, then locate the named textbook chapter, slides, lesson plan, or attachment.
3. Local extraction: save originals under readings/; extract searchable text from PDFs and other readable files beside them. Preserve page/source anchors.
4. Public-source fallback: only when the assigned text remains unavailable, search by exact title, author and book. Prefer the original text, publisher, author, library or other primary source. Save reconstructed notes under references/ and label their confidence as medium or low.

If no sufficiently authoritative source remains after all four layers, write error with reason_code=required_reading_unavailable; do not guess answers.

Build study_notes.md from the retrieved evidence. For each reading include:

• central thesis;
• key claims with page, paragraph, slide or URL anchors;
• names, dates, places and concepts;
• likely true/false or distinction targets;
• confidence and any source weakness.

For multiple readings, add cross-cutting themes. Put the overlay's instructor framework primer at the top, clearly labeled as course context, and keep direct source claims distinguishable from inference.

3. Preflight every mutation gate

After study_notes.md exists, enforce the four scheduling gates:

1. CANVAS_QUIZ_AUTORUN=1; otherwise return draft_ready with draft_path=study_notes.md.
2. Current America/Los_Angeles hour is inside CANVAS_QUIZ_HUMAN_HOURS or the overlay window; otherwise return the same draft_ready form.
3. runs/_processed.json contains fewer submitted quiz results in the prior six hours than CANVAS_QUIZ_MAX_PER_RUN or the overlay limit; otherwise return the same draft_ready form.
4. The exact course is in whitelisted_course_ids; otherwise write skipped.

Scheduling flags and a whitelist are not mutation authority. Require the signed, unexpired authorization_receipt_path supplied by canvas-submit or an authorized delegation; the interactive default is <work_dir>/mutation_authorization.json. It must be bound to the current Canvas origin, course, target_type="quiz", exact target_id=quiz_id, current Codex session, and exact action set. Validate every anticipated action through src.authorization.load_authorization_receipt and validate_authorization_receipt before starting, so a later scope failure cannot waste an attempt.

Attempt 1 therefore needs quiz.start, quiz.event, quiz.answer, and quiz.complete. Add quiz.retake only when the student's exact authorized workflow includes another attempt. No action implies another, and no wildcard, environment boolean, plan decision, overlay sentence, or prior receipt may replace an exact action. If the receipt is absent, invalid, expired, origin- mismatched, target-mismatched, session-mismatched, or missing a required action, write draft_ready with draft_path=study_notes.md and make no mutation.

Pass the same validated receipt to every client mutation. The client remains the authoritative enforcement boundary.

4. Open attempt 1 and collect questions

Start only after the entire attempt-1 scope has passed preflight:

sub = cv.start_quiz_submission(
    course_id, quiz_id,
    authorization_receipt=authorization_receipt,
)

Save the submission id, attempt number, validation token, end_at, and response clock to submission.json. Post one session_started event using the quiz.event scope. Fetch questions only from the student submission endpoint:

questions = cv.get_quiz_submission_questions(submission_id)

Save questions.json. Never bulk-post question_viewed events at open; pair each view with its later answer. Immediately after start, atomically update quiz_meta.json with course_id, quiz_id, assignment_id, the validated receipt's session_id and receipt_id (stored as authorization_receipt_id), and attempt=sub["attempt"]. These values bind all later local evidence to this one open attempt.

5. Normalize every supported question type

Strip HTML for reasoning while retaining raw question and answer identifiers. Write questions_simplified.json. Preserve these answer shapes exactly:

For blank questions, union prompt [variable_name] tokens with every returned answers[].blank_id; those variable names are the submission keys. Keep dropdown options grouped by blank. Never paste raw reading text into an essay.

6. Run four independent native Codex subagents in parallel

Spawn four separate native Codex subagents in one parallel dispatch, before awaiting any one result. Give each study_notes.md, normalized questions and the source files, but do not give it another pass, a proposed final answer, or the expected disagreement.

• notes-first: answer from study_notes.md, then verify uncertain claims in the exact source.
• grep-first: search the full extracts for every question before using general knowledge; cite a source anchor.
• framework-aware: use direct readings first and the overlay primer only for explicit lecture/framing questions.
• contrarian: challenge traps, negations, restrictive words, matching pairings and every option in a multiple-answer question.

Require each subagent to return only a JSON array. Every entry includes qnum, question_id, type, the correctly shaped answer, confidence, a concise reasoning, and source_anchor. Short and blank answers use the most likely accepted token; alternatives belong in reasoning, never in the answer value.

Preserve each returned array verbatim under answers in an evidence envelope whose context exactly repeats quiz_meta.json's six binding fields and whose agent_role names that pass. Save the four envelopes immediately as:

agent_passes/notes_first.json
agent_passes/grep_first.json
agent_passes/framework_aware.json
agent_passes/contrarian.json

They must be valid, independently produced JSON files. Do not synthesize four personas in one response, clone a file, or manufacture disagreement. All four may honestly choose the same answers when their independent reasoning and source checks support that consensus.

7. Arbitrate and write canonical evidence

Tabulate all four passes per question:

• 4-0 agreement: accept it;
• 3-1: take the majority and record the dissent;
• 2-2: resolve from an exact source anchor, wording and course framework;
• differing multiple-answer sets: verify every option separately.

Write final_answers.json before any answer mutation:

{
  "context": {
    "course_id": "12",
    "quiz_id": "34",
    "assignment_id": "56",
    "session_id": "<current Codex session>",
    "attempt": 1,
    "authorization_receipt_id": "<receipt id>"
  },
  "arbitration_notes": {
    "unanimous_count": 4,
    "flagged_qnums": [3],
    "Q3": "2-2 split resolved from source anchor ..."
  },
  "answers": [
    {
      "qnum": 1,
      "question_id": 101,
      "type": "multiple_choice_question",
      "answer": 1001,
      "confidence": "high",
      "source_anchor": "reading-a.txt paragraph 8"
    }
  ]
}

The integer arbitration_notes.unanimous_count, the exact current-attempt context, and at least four JSON files in agent_passes/ are required by src.canvas_client._require_canonical_arbitration_evidence. Answer consensus is valid; four canonically identical substantive arrays (answers plus reasoning) are copy-paste evidence and fail. If honest evidence cannot satisfy that guard, write error; never forge it. A degraded method is allowed only when CANVAS_QUIZ_DEGRADED_OK contains the student's verbatim, specific consent of at least ten non-space characters, and that same text is recorded as degraded_method_user_consent. It bypasses only the arbitration evidence guard, never the signed mutation receipt.

Use src.quiz_focus_events.pick_flagged_questions for a capped subset of low/medium-confidence questions. The optional src.quiz_strategic_miss.maybe_flip_answers branch runs only when CANVAS_QUIZ_STRATEGIC_MISS=1; never flip a high-confidence or constructed- response answer and retain its full log. It does not grant mutation authority.