Tldraw Offline

tldraw offline Skill

Work with the tldraw offline desktop app (offline.tldraw.com): read the open canvas, make edits, and write document scripts — JavaScript embedded in a .tldraw file that runs on load and gives the file durable behavior. The app runs a local HTTP API (default localhost:7236) that a coding agent drives with plain curl from its terminal — this is exactly how the app's own homepage demo (Codex editing a canvas live) works. The agent does NOT use computer-use / GUI clicking, and does NOT hand-edit the .tldraw file directly. Keep tldraw offline open while you work.

When to Use

• The user has tldraw offline open and asks you to build or modify a canvas (diagrams, wireframes, layouts).
• You want to add durable behavior to a drawing (reactive shapes, interactive buttons, animation, connection logic) via an embedded document script.

Do NOT hand-place shapes to imitate a drawing — write the code that generates them. Agents are far better at scripting the canvas than at drawing on it.

Prerequisites

• tldraw offline installed and running, with a document open. Releases: https://github.com/tldraw/tldraw-offline/releases/latest (macOS DMG, Windows x64/Arm64, Linux x86_64/arm64 AppImage or amd64/arm64 .deb).
• Agent skills installed in the app: Develop → Install Agent Skills. The app writes its own tldraw skill into ~/.codex/skills/, ~/.claude/skills/, ~/.cursor/skills/, and ~/.gemini/skills/ — teaching that agent the curl recipes below. (This Hermes skill mirrors that guidance for Hermes.)
• The local control API. On launch the app writes server.json to its config dir (Linux ~/.config/tldraw/, macOS ~/Library/Application Support/tldraw/, Windows %APPDATA%\tldraw\) with port (default 7236), a bearer token, pid, and startedAt. Every request except GET / needs Authorization: Bearer <token>. A clean quit removes server.json; if it's present but the port doesn't answer, the app quit uncleanly — treat as not running.
• Re-read port + token on EVERY shell call. Each terminal call is a fresh shell, so an exported token does not persist — "export once and reuse" sends an empty token and 401s. Read both inline at the top of each call: PORT=$(jq -r .port <server.json>); TOKEN=$(jq -r .token <server.json>).
• No account or network needed for local editing.

How to Run

Two distinct workflows. Pick by whether the change must survive a reload.

A. One-off canvas edits (/exec) — layout, generating shapes, cleanup. This is a live edit, not saved script:

BASE=http://localhost:7236
TOKEN=$(python -c "import json;print(json.load(open('$HOME/.config/tldraw/server.json'))['token'])")
# find the focused document id
DOC=$(curl -s "$BASE/api/search" -X POST -H 'content-type: application/json' \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"code":"return (await api.getFocusedDoc()).id"}' | python -c "import sys,json;print(json.load(sys.stdin)['result'])")
# run code with the live `editor` + `helpers` in scope
curl -s "$BASE/api/doc/$DOC/exec" -X POST -H 'content-type: application/json' \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"code":"const {createShapeId,toRichText}=await import(\"tldraw\"); editor.createShape({id:createShapeId(),type:\"geo\",x:0,y:0,props:{geo:\"rectangle\",w:200,h:100,color:\"blue\",fill:\"solid\",richText:toRichText(\"hello\")}}); return editor.getCurrentPageShapes().length"}'

B. Durable behavior (script/main.js) — reactive/interactive logic that must survive reload. Edit the file on disk; the app's watcher applies it:

# get the live script file path for the doc
curl -s "$BASE/api/doc/$DOC/script-workspace" -X POST \
  -H "Authorization: Bearer $TOKEN"          # -> result.mainJsPath, result.isDefaultScript
# edit result.mainJsPath with read_file / patch / write_file (see scripts/main.js)
# then confirm the watcher applied it:
curl -s "$BASE/api/doc/$DOC/script-status" -H "Authorization: Bearer $TOKEN"

The ready-to-adapt document script is scripts/main.js.

Quick Reference

The document-script contract (verified against the app's bundled script-context.d.ts):

import { createShapeId, toRichText } from 'tldraw'   // primitives: import, not globals

export default function ({ editor, helpers, signal }) {
  editor.run(() => {                                 // batch = one undo step
    helpers.createShapeIfMissing({                   // idempotent furniture
      id: createShapeId('node-1'), type: 'geo', x: 0, y: 0,
      props: { geo: 'rectangle', w: 200, h: 100, richText: toRichText('hi') },
    })
  })

  const stop = editor.store.listen(() => { /* react */ })  // fires the tick AFTER a commit
  signal.addEventListener('abort', () => stop())           // REQUIRED cleanup on rerun/close
}

• ctx.editor — the live Editor (createShape, updateShape, deleteShapes, getCurrentPageShapes, getShape, getBindingsFromShape, zoomToFit, on('tick'|'event', fn), run(fn, { history: 'ignore' })).
• ctx.helpers — createShapeIfMissing, createShapesIfMissing, createArrowBetweenShapes(from, to, { arrowheadEnd }), translateShapes, onShapeTranslate(id, fn, { signal }), richTextToPlainText, boxShapes, getLints.
• ctx.signal — AbortSignal; attach every listener/interval teardown to it.
• config.js (separate file) registers custom shape/tool/component utils and runs before mount; main.js runs against the mounted editor and reruns on save.

Interactive UI (clickable buttons that drive state)

Drawn shapes can behave like a real app — the thing a static whiteboard can't do. Full example: scripts/counter.js (a number display + MINUS/RESET/PLUS buttons).

Verification boundary — read this before claiming interaction works or doesn't. The app's OWN agent playbook says to verify a clickable-UI script with "one simulated click and one state read" via /exec (editor.dispatch a pointer event, await a tick, read the shape's state) — NOT by driving a real mouse. By that standard the counter is verified: dispatched clicks stepped it 0 → 1 → 2 → 1 → 0. Two caveats worth writing down:

• The script only runs once the app's file-watcher applies it. On Linux that watcher uses inotify; a host with an exhausted fs.inotify.max_user_instances logs inotify_add_watch ... No space left on device, script-status shows state: "not-watching" / hasEntry: false, and the script never executes. This is a host limit, not a script bug — a normal desktop is unaffected.
• Driving the canvas with computer-use needs FOREGROUND delivery, not background. cua-driver's default background delivery returns background_unavailable against Electron's occluded renderer — but that is the first rung, not a wall. cua-driver returns an escalation: "foreground" hint; set delivery_mode: "foreground" (pair with bring_to_front) and its X11 XTest path (x11_xtest_fg) clicks land — this dismisses the "Run Script" consent dialog and clicks the canvas. Verified by the Cua team on this exact v1.11.0 AppImage (Linux/X11). Do NOT conclude "Electron rejects synthetic clicks" and give up in background mode — climb to foreground. (The real product path is still /exec, not clicks; this note is for computer-use-driven testing.)

The pattern:

export default function ({ editor, helpers, signal }) {
  // 1. Build buttons idempotently; tag each with meta so the handler finds them.
  //    Give buttons a visible label AND a meta.action.
  // 2. Hit-test pointer_down in PAGE coordinates against the button bounds:
  const inside = (b, p) => p.x >= b.x && p.x <= b.x + b.w && p.y >= b.y && p.y <= b.y + b.h
  function onEvent(info) {
    if (!info || info.name !== 'pointer_down') return
    let p = null
    try { if (info.point && editor.screenToPage) p = editor.screenToPage(info.point) } catch {}
    p = p ?? editor.inputs?.currentPagePoint
    if (!p) return
    const hit = editor.getCurrentPageShapes().find(
      (s) => s.meta?.ui === 'button' &&
        inside({ x: s.x, y: s.y, w: s.props.w, h: s.props.h }, p)
    )
    if (hit) runAction(hit.meta.action)   // mutate state; store it in a shape's meta
  }
  editor.on('event', onEvent)
  signal.addEventListener('abort', () => editor.off('event', onEvent))  // REQUIRED
}

• Find buttons by meta (or visible label via helpers.richTextToPlainText), not by hard-coded coordinates.
• One script owns both build and read. If the shapes are created by one code path (with meta.action: 'inc') and the handler reads another convention (meta.action === 'PLUS'), clicks silently do nothing. Ship the buttons built by the same script that handles them, or ship an empty canvas so the script builds them fresh — never pre-bake mismatched shapes into the file's db.
• Keep app state in a shape's meta (e.g. meta.count) and render it as that shape's richText label, so it survives save and is readable for verification.
• Detach the listener on signal abort. Skipping this is not cosmetic: on the next save the old onEvent stays attached alongside the new one, so every click fires twice and a counter jumps by 2 instead of 1.
• For continuous motion use editor.on('tick', fn); for a moving anchor with attached pieces use helpers.onShapeTranslate(id, fn, { signal }).