Daemon Dev

Daemon Development

Quick Reference

Architecture

The daemon (runtimed) is a singleton coordinating notebook windows over a Unix socket. Without it: race conditions on prewarmed environments, wasted resources, slow cold starts.

Components: Unix socket (IPC), lock file (singleton guarantee), UV/Conda pools (prewarmed envs), blob store (content-addressed outputs), notebook-docs (persisted Automerge documents).

Protocol: Length-prefixed typed frames over Unix socket. Preamble → JSON handshake → Automerge sync → steady state. Frame types: AutomergeSync (0x00), Request (0x01), Response (0x02), Broadcast (0x03), Presence (0x04), RuntimeStateSync (0x05), PoolStateSync (0x06), SessionControl (0x07), PutBlob (0x08), CommsDocSync (0x09).

CRDT ownership: Frontend WASM writes cell source, position, type, metadata. Daemon writes RuntimeStateDoc for outputs, execution counts, comm topology, trust, env progress, and project context. Mutable widget state lives in CommsDoc and is gated by RuntimeStateDoc topology. Write to a CRDT document only from an authority allowed for that document.

Build System

How cargo xtask build works (4 phases)

0. Artifact guard: Verify gitignored WASM, renderer-plugin, and MCP widget outputs. Build/dev commands fingerprint workspace inputs and skip wasm-pack when outputs are current; rebuild only when outputs are missing, invalid, or stale.
1. Single Rust compilation: cargo build -p runtimed -p runt -p nteract-mcp -p mcp-supervisor. Sidecars copied to crates/notebook/binaries/. The notebook crate is intentionally excluded until the Tauri link phase after frontend assets exist.
2. Frontend build: pnpm build (TypeScript + Vite). --rust-only skips this.
3. Tauri link: cargo tauri build --debug --no-bundle with embedded frontend assets. Use --skip-tauri only for fast edit checks where updated sidecar binaries are enough; run a normal build before launching the bundled app.

All Rust targets build in one cargo build call to avoid feature-unification recompilation. WASM outputs are gitignored; runtimed's build.rs panics if missing. nteract-mcp embeds the MCP widget HTML; prepare it with cargo xtask artifacts ensure mcp-widget.

WASM rebuild

Run after changing crates/runtimed-wasm/, crates/sift-wasm/, crates/notebook-doc/, crates/notebook-wire/, or scripts/build-renderer-plugins.ts:

cargo xtask wasm             # all (runtimed-wasm + sift-wasm + renderer plugins)
cargo xtask wasm runtimed    # only runtimed-wasm
cargo xtask wasm sift        # only sift-wasm

Development Workflow

Fast iteration

cargo xtask dev

This is the normal desktop workflow: it starts the per-worktree daemon, waits for readiness, launches the hot-reload app, and stops the daemon it started when the app exits. On macOS the app has a distinct per-worktree display name, bundle ID, and process record. Inspect or focus that exact process with:

cargo xtask dev status
cargo xtask dev focus

After Tauri bootstrap or listener changes, use cargo xtask dev --fresh to reset only this worktree's app and daemon. Use the split workflow when the daemon must remain independently managed:

# Terminal 1
cargo xtask dev-daemon

# Terminal 2
cargo xtask notebook

Testing

cargo test -p runtimed                          # All tests
cargo test -p runtimed --test integration       # Integration tests only
cargo test -p runtimed test_daemon_ping_pong    # Specific test

Per-worktree isolation

cargo xtask dev, cargo xtask dev-daemon, cargo xtask notebook, and cargo xtask run-mcp derive the git worktree automatically. State lives at <cache>/runt-nightly/worktrees/{hash}/. Set RUNTIMED_DEV=1 and RUNTIMED_WORKSPACE_PATH="$(pwd)" only for raw ./target/debug/runt commands outside xtask.

Python Bindings

Two venvs

Installation

# Into workspace venv (most common; what `up rebuild=true` does)
cd crates/runtimed-py
VIRTUAL_ENV=../../.venv uv run --directory ../../python/runtimed maturin develop

# Into test venv (for pytest)
VIRTUAL_ENV=../../python/runtimed/.venv uv run --directory ../../python/runtimed maturin develop

Always set VIRTUAL_ENV explicitly. Without it, the .so installs into whichever venv uv run resolves.

Basic usage

import asyncio, runtimed

async def main():
    client = runtimed.Client()
    async with await client.create_notebook() as notebook:
        cell = await notebook.cells.create("print('hello')")
        result = await cell.run()
        print(result.stdout)  # "hello\n"

        # Granular execution control
        execution = await cell.execute()
        print(execution.status)  # "queued" | "running" | "done" | "error"
        result = await execution.result()

asyncio.run(main())

Output.data typing

Integration tests

RUNTIMED_SOCKET_PATH="$(./target/debug/runt daemon status --json | python3 -c 'import sys,json; print(json.load(sys.stdin)["socket_path"])')" \
  python/runtimed/.venv/bin/python -m pytest python/runtimed/tests/test_daemon_integration.py -v

MCP Server

Use nteract mcp for the supervised MCP server. runt mcp remains available to run the worker directly. Run via cargo xtask run-mcp for development.

Advertised tools (all_tools()): list_active_notebooks, list_notebooks, connect_notebook, create_notebook, save_notebook, show_notebook, disconnect_notebook, create_cell, set_cell, delete_cell, move_cell, execute_cell, run_all_cells, get_results, interrupt_kernel, restart_kernel, manage_dependencies, replace_match, replace_regex.

Hidden/callable read tools (hidden_tools()): get_cell, get_all_cells. These are callable but not advertised in list_tools(). They are dispatch-only read paths for clients that prefer direct tool calls over resource URIs.

Legacy dependency and cell-metadata tool names still dispatch for compatibility, but new workflows should use manage_dependencies for dependency inspection/edits and get_results for execution output lookup by execution_id.