Automerge Sync

Automerge Sync & Document Model

Dependency baseline (2026-09-04)

Production Rust and runtimed-wasm use crates.io Automerge exactly 0.11.0 (Cargo.toml:57, Cargo.lock:627). Commit ae6aef0f adopted it on 2026-08-26. The frontend uses these Rust WASM bindings, not @automerge/automerge.

The only remaining Automerge git dependency is automerge-legacy, a dev-dependency of automerge-store pinned to nteract/automerge revision 3fb6af5cc3af23b79f27cebfa339c8c98987e7b7 (Rust 0.10.0). It is a test peer for snapshot and encoded-sync compatibility, not a production patch override. See crates/automerge-store/Cargo.toml:17 and crates/automerge-store/tests/version_compat.rs:282.

Document Model Essentials

Core Types

Actor table ordering: OpSet.actors is a sorted Vec<ActorId>. Ops store only the index. If two documents disagree on index→actor mapping, ops are misinterpreted. This is the root of the historical #1187 panic class.

The Automerge / AutoCommit Structs

// Automerge: raw document
Automerge { queue: ChangeQueue, change_graph: ChangeGraph, deps: HashSet<ChangeHash>, ops: OpSet, actor: Actor }

// AutoCommit: wrapper nteract uses
AutoCommit { doc: Automerge, transaction: Option<(PatchLog, TransactionInner)>,
             patch_log: PatchLog, diff_cursor, save_cursor, isolation: Option<Vec<ChangeHash>> }

Auto-transaction: Mutations open a transaction implicitly. Reads, save(), fork(), merge(), and sync() commit pending ops first.

Isolation mode: isolate(heads) limits visible state. Mutations while isolated depend on isolation heads, not tips. integrate() returns to latest.

PatchLog: Tracks diffs for incremental materialization. nteract's WASM computes CellChangeset by diffing after sync frames.

save() and load()

• save() serializes OpSet columns + ChangeGraph metadata + optional DEFLATE. Columnar format is canonical.
• load() rebuilds OpSet from columns, reconstructs ChangeGraph, verifies heads.
• save/load round-trip rebuilds indices from serialized state. nteract's rebuild_from_save() can fail or reject cell loss; it is containment, not a guarantee that arbitrary corruption is repaired.
• load_incremental() adds changes to existing doc. This is what receive_sync_message calls internally.
• save_after(heads) emits only changes after given heads (incremental saves).

Fork and Merge

DuplicateSeqNumber trap: Two concurrent forks sharing the same ActorId produce changes with identical (actor, seq). The second merge fails. Use unique actors for concurrent forks.

Document Size

save() compacts via columnar + DEFLATE. No history compaction exists.

Sync Protocol Internals

Sync Message Structure

Message { heads, need, have: Vec<Have>, changes: ChunkList, flags, version }

• heads: "here's what I have"
• need: "I'm missing these specific changes"
• have: bloom filter (1% FP rate, 10 bits/entry, 7 probes) of changes since last_sync
• changes: actual change data
• Sender sends a change only if all peer bloom filters say they lack it

sync::State: Per-Peer Session

Critical: encode() only serializes shared_heads. All else is session-ephemeral. sync::State::new() is always safe for reconnection: you lose optimization (may resend) but keep correctness.

In-Flight Suppression

generate_sync_message() returns None when in_flight && last_sent_heads == our_heads && have_responded. Any incoming message sets in_flight = false (counts as ack). If you need a fresh exchange, reset sync state rather than working around None.

Change Selection

1. Compute needed deps from their advertised heads
2. Build bloom filter from our changes since shared_heads
3. Filter through their bloom (send what they probably lack), deduplicate against sent_hashes
4. If sending >1/3 of doc, send whole doc as V2 (more efficient)

Version Negotiation

V1 is original; V2 allows compressed document encoding. Backward-compatible via MessageFlags appended to V1 messages. V2 discovered via flags, then used for subsequent messages.

nteract Sync Architecture

Document Streams Over One Socket

For CommentsDoc, see crates/comments-doc, daemon persistence at crates/runtimed/src/notebook_sync_server/comments_store.rs, ingress at peer_comments_sync.rs. Typed clients (notebook-sync crate) and frontend WASM (runtimed-wasm) both hold CommentsDoc replicas. Optimistic client mutations apply via Automerge; the daemon validates change actor labels against the connection principal (clone-preview) and strips writes from scopes without comment authority. There is no daemon finalization step. Attribution (resolved_by_actor_label, resolved_at) is projected from admitted change actors.

Sync Task Loop (biased select!)

Priority: Frame (drain socket) → Changed (outbound sync) → Command (RPC) → Maintenance (50ms tick).

Mutex is std::sync::Mutex, never held across .await. Poison recovery: unwrap_or_else(|e| e.into_inner()).

Document-Level Recovery

Automerge is treated as a fallible boundary. Policy belongs to the document owner; sync and mutation helpers do not handle panics the same way.

Sync policy (2026-09-04):

• recoverable_automerge_operation rebuilds and retries once only for a caller-marked operation error. It returns a caught panic immediately, without rebuild or retry (crates/automerge-recovery/src/lib.rs:163–201).
• NotebookDoc::receive_sync_message_recovering marks only AutomergeError::PatchLogMismatch(_) recoverable. It resets the peer's sync::State, rebuilds the document, then retries the original frame once (crates/notebook-doc/src/lib.rs:2479–2508, crates/automerge-recovery/src/lib.rs:159–160).
• NotebookDoc::generate_sync_message_recovering marks no operation error recoverable; caught panics are returned to the caller (crates/notebook-doc/src/lib.rs:2446–2464).

Mutation policy: NotebookDoc::merge_recovering attempts to rebuild both sides after a panic, then returns the failure. transact_at_heads_recovering restores actor/isolation state and attempts rebuild on panic without rerunning the mutation closure (crates/notebook-doc/src/lib.rs:400–491). RuntimeStateDoc has its own transaction recovery and rollback handling (crates/runtime-doc/src/doc.rs:761–819). These are nteract wrappers around upstream APIs, not fork-only methods.

Notebook rebuild: rebuild_from_save saves and loads the document, rejects a result with fewer cells, and preserves the actor before replacing the live document (crates/notebook-doc/src/lib.rs:1477–1502). Rebuild can fail. Peer state reset belongs to the calling sync helper, not to rebuild_from_save. Do not treat save/load as proof that arbitrary corruption has been repaired.

Causal Ordering: required_heads (preferred)

1. Client captures current heads via DocHandle::current_heads_hex()
2. Sends request with required_heads in envelope
3. Daemon's wait_for_required_heads() checks containment via get_change_by_hash
4. Defers processing until all heads arrive (10s timeout) or proceeds immediately
5. Sync loop stays unblocked; only that specific request waits

confirm_sync (legacy alternative): Client-side waiter on shared_heads. Blocks client, daemon free. Still used for SaveNotebook.

RuntimeStateDoc Output Pressure

RuntimeStateDoc is the durable state boundary, not the hot transport for every transient kernel event. Control-plane signals must stay independent of output work:

• KernelIdle, ExecutionDone, CellError, and KernelDied use reliable lifecycle/control paths, not bounded output queues.
• stdout/stderr stream chunks may be periodically flushed through bounded, droppable work, but ordering boundaries use the stream committer priority path so terminal state follows the final durable stream manifest.
• Output widget replay back to the kernel is best-effort; widget state in RuntimeStateDoc is the durable truth.
• update_display_data with a display_id is transient display churn. Coalesce to the latest pending value per display_id off the IOPub path, then flush before ExecutionDone.