Canvas Humanizer

canvas-humanizer v2 — round-trip translation + deterministic divergence search

§1 — Identity & contract (caller-facing, identical to v1)

Inputs (parsed from caller's prose context line; pattern-matching is the same as v1):

If the caller's prose context omits student_identity, v2 silently uses voice_register as the identity. This restores backward compatibility with v1 callers that may have hardcoded only three args.

Outputs:

• output_path — humanized draft
• <output_dir>/humanizer_log.json — per-segment trace (candidates, scores, language paths)

Status return:

• ok — every segment cleared the meaning+voice gate AND structural divergence > 0.30
• partial — at least one segment had to fall back to original because no candidate passed meaning gate
• error — input validation failure (missing file, missing arg, etc.)

Caller compatibility: input/output contract is identical to v1. canvas-essay §7.5 invocation works unchanged.

§2 — Why v2 (read this once, then never again)

v1 failed in production with a Turnitin AI-detection result of 75% AI despite v1's internal 3D audit reporting status: ok. Five root causes (do not re-discuss in v2 SKILL.md — they are settled):

1. LLM-as-judge is self-referential. v1 used Claude/GPT to score Claude/GPT-generated text. Audit pass ≠ Turnitin pass.
2. ±5% word-count clamp was a structural straitjacket. Real humanization needs sentence restructuring, not lexical substitution.
3. Banned-words list is cosmetic. AI fingerprint is in token-probability distributions, not in 30 high-frequency words.
4. 3 sequential passes (vocab → sentence → texture) all happen inside the LLM-English distribution. Each pass stays in the mode it's trying to escape.
5. No adversarial feedback loop. v1 couldn't query the actual detector, so it had no idea whether it was getting closer to or further from passing.

v2's response to each:

§3 — Pipeline overview

Input docx/md
  ↓
[§4] Identify preservation locks   →  list of verbatim spans that NEVER round-trip
  ↓
[§5] Segment to sentences          →  flat list of segments, each marked humanizable/locked
  ↓
[§6] Round-trip K candidates       →  per humanizable segment, K=3 round-trips via different intermediate languages
  ↓
[§7] Score candidates              →  for each: (meaning_preserved × voice_intact × divergence) ; pick max
  ↓
[§8] Reassemble                    →  substitute locks back, glue segments into paragraphs preserving doc_index
  ↓
[§9] (Optional) Detector loop      →  if overlay's detector_api != none, call adapter; iterate from §6 with deeper perturbation if score > target
  ↓
[§10] Write artifacts              →  output_path + humanizer_log.json

§4 — Preservation locks

Some spans MUST survive every round-trip byte-for-byte:

• Direct quotes — anything between "..." (or “...”) that matches the source text
• Named entities — author names, place names, proper nouns the caller supplies
• Numbers + dates — 38 percent, 1982, 22 April 2026, two to three percent
• Caller-supplied hard locks — overlay field hard_locks (per-essay list)

§4a — Lock identification

Run a Python helper to extract candidate locks:

import re, json
from pathlib import Path

def extract_locks(text: str, hard_locks: list[str]) -> list[tuple[int, int, str]]:
    """Return list of (start_char, end_char, span_text) for spans that must not be touched.
    Order: leftmost first; non-overlapping (longer wins on conflict)."""
    locks = []
    # 1. Quoted spans (straight or curly)
    for m in re.finditer(r'"([^"]+)"|"([^"]+)"', text):
        s, e = m.span()
        locks.append((s, e, text[s:e]))
    # 2. Years 1900-2099
    for m in re.finditer(r'\b(19|20)\d{2}\b', text):
        locks.append((m.start(), m.end(), m.group(0)))
    # 3. Percent literals (38 percent, 2 to 3 percent, etc.)
    for m in re.finditer(r'\b\d+(?:\s*(?:to|and)\s*\d+)?\s+percent\b', text, re.I):
        locks.append((m.start(), m.end(), m.group(0)))
    # 4. Dates like "22 April 2026" / "April 22, 2026" / "May 22"
    for m in re.finditer(r'\b\d{1,2}\s+(January|February|March|April|May|June|July|August|September|October|November|December)(?:\s+\d{4})?\b|\b(January|February|March|April|May|June|July|August|September|October|November|December)\s+\d{1,2}(?:,\s*\d{4})?\b', text):
        locks.append((m.start(), m.end(), m.group(0)))
    # 5. Age-cohort terms (Gen Z, Gen X, millennials, etc.) — Claude routinely
    #    translates "Gen Z" → "Generation Z" / "Z 世代" without the agent NE pass
    #    catching it. Regex coverage prevents this.
    for m in re.finditer(r'\b(Gen\s*(?:Z|X|Y|Alpha)|Generation\s*(?:Z|X|Y|Alpha)|Millennials?|Boomers?|Gen\s*Z-ers|Gen-Zers)\b', text, re.I):
        locks.append((m.start(), m.end(), m.group(0)))
    # 6. Caller-supplied hard locks (search literally)
    for span in hard_locks:
        for m in re.finditer(re.escape(span), text):
            locks.append((m.start(), m.end(), m.group(0)))
    # Deduplicate + resolve overlaps: sort by start asc, length desc; greedy keep
    locks.sort(key=lambda t: (t[0], -(t[1]-t[0])))
    out, last_end = [], -1
    for s, e, sp in locks:
        if s >= last_end:
            out.append((s, e, sp))
            last_end = e
    return out

§4b — Named-entity locks via Agent

After regex extraction, spawn ONE agent to find named-entity locks the regex missed:

Task: list every proper noun OR domain-specific term in this text that should be preserved verbatim across translation. Return a JSON list of exact strings as they appear in the text. Include:

• Person names (e.g., [Author A], [Author B])
• Organizations (e.g., Government Accountability Office, Federal Trade Commission)
• Publications (e.g., [Publication], Times Opinion)
• Geographic locations (e.g., Shanghai, Oregon)
• Brand / product / platform names (e.g., Etsy, YouTube, Canvas)
• Age-cohort terms (e.g., Gen Z, Gen X, Millennials, Boomers) — these are commonly translated/expanded into Generation Z / Z 世代 by translators and must be locked
• Technical / discipline-specific terms (e.g., Current Population Survey, noncompete agreements)

Do NOT include common nouns, generic abstractions, or verbs. Do NOT paraphrase entries. Return at most 25 items as a JSON array of strings.

Text:

{text}

Add the returned strings to hard_locks before §4a's pass.

§4c — Mask placeholders

Replace each lock span with [LOCK_N] where N is the lock index (0-based). The masked text passes to translation; the lock list passes alongside. After round-trip, substitute [LOCK_N] → exact original bytes.

§4d — Order of operations: split THEN mask, not mask THEN split

Important (per 2026-05-22 full-essay run): do the §5 sentence splitting on the unmasked text first, then mask each sentence with paragraph-global lock indices. The reverse order — mask-then-split — fails when a lock span absorbs sentence-ending punctuation (e.g. a quote ending in ." followed by This finding...), because the splitter can no longer find the sentence boundary inside [LOCK_N]. Always split first, mask second, but keep lock indices consistent across the whole paragraph (don't reuse index 0 in two different sentences).

§5 — Sentence segmentation

Default level: sentence. Overlay can override to clause or paragraph.

§5a — Sentence tokenizer

Use Python regex (avoid pulling NLTK/spaCy — zero-dependency policy):

def split_sentences(text: str) -> list[str]:
    """Conservative sentence splitter: handles common abbreviations, decimals, ellipses."""
    # Protect common abbreviations + numeric decimals + ellipses
    protected = text
    abbrevs = ['Mr.', 'Mrs.', 'Ms.', 'Dr.', 'Prof.', 'Inc.', 'Ltd.', 'St.', 'Jr.', 'Sr.', 'vs.', 'e.g.', 'i.e.', 'etc.', 'Co.', 'U.S.', 'U.K.']
    for ab in abbrevs:
        protected = protected.replace(ab, ab.replace('.', '\x00'))
    # Protect decimals (3.14, 1,000.50). Use lambda — a raw-string replacement r'\1\x00\2'
    # would inject the LITERAL 4-char sequence \x00, not a NUL byte, because Python raw
    # strings don't interpret \x escapes. Lambda lets us insert a real NUL.
    protected = re.sub(r'(\d)\.(\d)', lambda m: m.group(1) + '\x00' + m.group(2), protected)
    # Protect ellipses
    protected = protected.replace('...', '\x00\x00\x00')
    # Split on .!? followed by whitespace+capital, OR end-of-string
    parts = re.split(r'(?<=[.!?])\s+(?=[A-Z"\'])', protected)
    # Unprotect
    parts = [p.replace('\x00\x00\x00', '...').replace('\x00', '.') for p in parts]
    return [p.strip() for p in parts if p.strip()]

§5b — Per-segment metadata

For each sentence:

{
  "seg_id": "S{n}",
  "doc_paragraph_index": <int>,   # which paragraph in docx
  "intra_para_index": <int>,      # 0-based position within paragraph
  "original": "<text>",
  "word_count": <int>,
  "contains_lock": <bool>,        # does any LOCK_N placeholder appear?
  "humanizable": <bool>,          # not entirely a lock, not too short (<5 words)
}

Skip segments where humanizable == False: name-block lines, title, short interjections, sentences that are ≥80% lock content.

§6 — Candidate generation (K candidates via TWO parallel strategies)

For each humanizable segment, generate K candidates (default K=6) via two parallel strategies:

• Strategy R (round-trip translation) — K_R candidates, each a round-trip through a different intermediate language. Perturbs prose via cross-lingual re-encoding. Default K_R = 3.
• Strategy P (structured English paraphrase) — K_P candidates, each applying a different deterministic structural transformation in English. Perturbs prose via explicit syntactic surgery. Default K_P = 3.

K = K_R + K_P. Both strategies feed the same downstream scoring (§7). Selection is "max divergence subject to meaning + voice preservation" across the unified pool, regardless of which strategy produced the candidate.

Honest framing (per §2's design review): round-trip via Claude itself is perturbed diversification, not true distribution escape — the intermediate Chinese/Japanese/German output is still rendered by the same model. Strategy R + Strategy P together act as diverse paraphrase generators; the deterministic Levenshtein selection in §7 is what gives the pool detector-relevant value.

Empirical observation (2026-05-22 full-essay run on 23 humanizable sentences): Strategy R won 0 of 23 times when round-trip is executed by Claude. Strategy P won 23 of 23 — clause_reorder dominated (13), with voice_flip (5) and nominalize_flip (5) splitting the rest. The honest read: in-Claude round-trip produces candidates that cluster too close to the original (divergence range 0.05-0.30), while structured English paraphrase produces explicit syntactic perturbation that consistently scores higher on Levenshtein. If real distribution escape is the target (e.g. trying to take Turnitin score from 75% to <20%), the round-trip hop needs an external engine (DeepL / Google Translate API) — the K_R bucket then becomes load-bearing. Until then, the K_R candidates are essentially "free additional variety" with low win rate; K_P is doing the actual humanization work.

Dispatch mechanism (clarified after 2026-05-22 smoke test):

• Preferred (parallel): when overlay has parallel_candidate_dispatch: true (the default), the orchestrator spawns the K candidates for one segment as K parallel Agent tool calls in a single message. This collapses ~5x wallclock and is the production-intended mode when canvas-essay §7.5 invokes the skill via Skill tool from a Claude Code session.
• Acceptable (inline): when the skill is being executed by a sub-agent that prefers in-context generation (e.g., a wrapping general-purpose agent doing a smoke test), the same prompts in §6b and §6d can be executed as direct LLM completions rather than spawned Agent calls. Outputs are identical; only parallelism is lost.

Both modes follow the same §6b/§6d prompt templates and the same §7 selection rule. Choose based on whether the executing context has the Agent tool available and a clear parallelism win.

§6a — Strategy R: round-trip translation

Language pool (overlay-overridable as languages_to_round_trip_through):

- zh         # Chinese — strong typological distance from English
- ja         # Japanese
- de         # German — close to English syntactically; gentle round-trip
- es         # Spanish — Latin family
- fr         # French
- ko         # Korean

For each candidate k_r in [0, K_R), pick language pool[k_r % len(pool)]. K_R=3 → uses zh, ja, de.

§6b — Translation prompts

Hop 1: EN → intermediate language

Spawn 1 Agent per candidate. Prompt:

Task: Translate the following English sentence to {language}. Constraints:

• Preserve every [LOCK_N] placeholder byte-for-byte. Do NOT transliterate, reorder, translate, or convert to characters in another script. The literal sequence [LOCK_N] (left bracket, the letters LOCK, underscore, digit, right bracket) must appear in your output exactly as in the input. Treat the [LOCK_N] token as if it were a code variable, not a translatable phrase.
• Preserve meaning faithfully.
• Do not add commentary, parentheticals, or footnotes.
• Output ONLY the translation, no preamble.

Sentence:

{masked_segment}

Hop 2: intermediate → EN (with register restoration)

Spawn 1 Agent per candidate. Prompt:

Task: Translate the following {language} sentence to English, in the register of {voice_register}.

Constraints:

• Preserve every [LOCK_N] placeholder byte-for-byte. Do NOT transliterate, reorder, translate, or convert to characters in another script. The literal sequence [LOCK_N] (left bracket, the letters LOCK, underscore, digit, right bracket) must appear in your output exactly as in the input. Treat the [LOCK_N] token as if it were a code variable, not a translatable phrase.
• Use natural English; do not produce a literal word-by-word back-translation.
• Voice register guidance for {voice_register}:
  • advanced-academic-english: long complex sentences, formal vocabulary, no contractions, no slang
  • b1-b2-international-student: simple sentence structures, occasional missing articles, present-tense leaning
  • (others fall back to native fluent English)
• Do not introduce banned words from this list: {banned_list}
• Output ONLY the English translation, no preamble, no quotation marks around the output.

Sentence:

{intermediate_text}

§6c — Substitute locks back

After hop 2, [LOCK_N] placeholders in the candidate get replaced with the exact original bytes from §4. If a candidate has lost a lock placeholder (translation broke it), discard that candidate and retry once with the EN → intermediate hop. If still broken after one retry, mark candidate as lock_lost and exclude from selection.

Per-segment lock check (per 2026-05-22 full-essay run): when checking "did this candidate lose any locks?", check only against the locks that appeared in this segment's masked original, not against the full paragraph's lock list. Paragraph-wide checks falsely fail candidates whose segment didn't contain a lock that other segments did. Compute required_locks_for_this_segment = {N for N in re.findall(r'\[LOCK_(\d+)\]', masked_segment_input)} and only verify those indices are present in the candidate output.

§6d — Strategy P: structured English paraphrase

For each candidate k_p in [0, K_P), apply one structural transformation to the original sentence. Each transformation is a deterministic LLM-prompted rewrite that targets a specific syntactic dimension.

Default transformation set (overlay-overridable as paraphrase_strategies):

For K_P=3, default transformations used are voice_flip, nominalize_flip, clause_reorder. Overlay can list a different subset.

Per-candidate prompt (one Agent spawn per K_P transformation):

Task: Rewrite the following English sentence by applying the {transformation_name} transformation:

{transformation_description}

Constraints:

• Preserve every [LOCK_N] placeholder byte-for-byte. Do NOT transliterate, reorder, translate, or convert to characters in another script. The literal sequence [LOCK_N] must appear in your output exactly as in the input.
• Preserve meaning faithfully. If the transformation is genuinely impossible for this sentence (e.g., voice_flip on an intransitive sentence), output the sentence unchanged.
• Voice register: {voice_register} — keep formal/informal register intact.
• Do not introduce banned words from this list: {banned_list}
• Output ONLY the rewritten sentence, no preamble, no quotation marks around the output, no explanation of what you did.

Sentence:

{masked_segment}

If the transformation prompt returns the unchanged input verbatim (transformation was impossible), mark candidate as paraphrase_inapplicable and exclude from §7 selection — but the candidate is not a failure, just a non-contribution to the pool.

Strategy P candidates do NOT round-trip through a non-English language. They are direct in-English rewrites. The candidate's value comes from the structural transformation being explicit (not "rewrite to be different"), producing reliably high Levenshtein divergence on the dimension targeted by the transformation.

§6e — Word-count gate per candidate (sliding tolerance)

Compute candidate_word_count / original_word_count. Tolerance scales by original sentence length — short sentences need wider bands because a single em-dash parenthetical or fronted adverbial clause adds 3-5 words that blow a tight band.

def word_count_tolerance(original_wc: int) -> tuple[float, float]:
    """Return (lower, upper) ratio band. Short sentences get wider bands."""
    if original_wc <= 15:
        return (0.70, 1.40)   # ±40% — covers clause_reorder parentheticals on short sentences
    elif original_wc <= 25:
        return (0.75, 1.30)   # ±30%
    else:
        return (0.80, 1.20)   # ±20% — original v2 default for longer segments

Outside band → discard candidate. Applies to both Strategy R and Strategy P candidates.

Rationale: smoke-test (2026-05-22 intro pass) found that on 25-word sentences, the highest-divergence candidates (clause_reorder, divergence 0.81 and 0.65) were disqualified by a fixed ±20% band, costing the pipeline its best perturbation. Sliding tolerance keeps the high-divergence candidates while still blocking truly runaway expansions.

§7 — Candidate scoring + selection (deterministic, no LLM judge)

For each surviving candidate, compute three scores:

§7a — meaning_preserved (LLM-judge, binary 0/1)

Spawn 1 Agent per (segment × candidate) pair. Prompt:

Task: Given the original sentence and a candidate rewording, return JSON {"meaning_preserved": true | false, "voice_register_intact": true | false, "rationale": "<one sentence>"}.

A meaning is preserved if the candidate makes the same claim about the same subject; minor surface differences (word order, synonyms) are fine. Voice register is {voice_register} — flag if candidate drifts to a noticeably different register (e.g. casual when advanced-academic expected).

Original:

{original}

Candidate:

{candidate}

Output strict JSON only.

This is the ONLY LLM-judge call in v2, and it's a binary semantic question (was meaning preserved?), not a graded AI-detection question. Self-reference risk is much lower because the judge isn't asked "does this look LLM-shaped?" — it's asked "is the meaning the same?".

§7b — voice_register_intact (same agent, same JSON response)

Returned by the §7a agent.

§7c — structural_divergence (deterministic)

Computed in Python — no LLM:

def levenshtein(a: str, b: str) -> int:
    """Standard DP edit distance, character-level."""
    if not a: return len(b)
    if not b: return len(a)
    prev = list(range(len(b)+1))
    for i, ca in enumerate(a, 1):
        cur = [i] + [0]*len(b)
        for j, cb in enumerate(b, 1):
            cost = 0 if ca == cb else 1
            cur[j] = min(cur[j-1]+1, prev[j]+1, prev[j-1]+cost)
        prev = cur
    return prev[-1]

def divergence(original: str, candidate: str) -> float:
    """0.0 = identical; 1.0 = totally different. Higher = more humanized."""
    if not original and not candidate: return 0.0
    L = max(len(original), len(candidate))
    return levenshtein(original, candidate) / L

divergence is in [0.0, 1.0]. We want it high (more humanized = more different from LLM-shaped original).

§7d — Final score + selection

def candidate_score(c) -> float:
    if not c["meaning_preserved"] or not c["voice_register_intact"]:
        return 0.0
    return c["divergence"]

Pick argmax. If max score == 0.0 (no candidate passed meaning gate), fall back to original sentence and mark segment as partial in the log.

Why this works: the selection rule is "biggest structural change subject to meaning preservation". v1 picked "passes LLM-judge audit". v2's selection signal is computable and adversarial-aware: Levenshtein distance is a property of the actual text, not an LLM's opinion of the text.