NanoClaw Container Debugging

NanoClaw Container Debugging

This guide covers debugging the containerized agent execution system.

Architecture Overview

The host is a single Node process that orchestrates per-session agent containers. The two session DBs are the sole IO surface between host and container — there is no IPC, no file watcher, and no stdin piping.

Host (Node)                                Container (Bun, Linux VM)
──────────────────────────────────────────────────────────────────────
src/container-runner.ts                    container/agent-runner/src/
    │                                          │
    │ spawns one container per session          │ polls inbound.db for work,
    │ with the session folder mounted          │ calls the agent provider,
    │ at /workspace                            │ writes replies to outbound.db
    │                                          │
    ├── data/v2-sessions/<group>/<session>/ ──> /workspace
    │     ├── inbound.db   (host writes, container reads RO)
    │     ├── outbound.db  (container writes, host reads)
    │     └── .heartbeat   (container touches → /workspace/.heartbeat)
    ├── groups/<folder> ─────────────────────> /workspace/agent  (cwd)
    ├── <group>/.claude-shared ──────────────> /home/node/.claude
    └── agent-runner src + skills ───────────> /app/src, /app/skills

Message flow: host writes a row to inbound.db (messages_in) and wakes the container; the container's poll loop picks it up, runs the agent, and writes the reply to outbound.db (messages_out); the host's delivery poll reads messages_out and sends it through the channel adapter. See docs/db.md and docs/db-session.md for the full two-DB model.

Container identity: the container runs as user node with HOME=/home/node. Per-group Claude state (settings, session history) lives in <group>/.claude-shared on the host, mounted to /home/node/.claude.

Log Locations

Containers run with --rm, so the container's own filesystem is gone after it exits. The host streams container stderr into logs/nanoclaw.log at debug level, tagged with container=<group folder>; raise the log level (below) to see it. If the agent silently failed inside an exited container, there is no persistent in-container log — reconstruct from the session DBs and the host log.

Enabling Debug Logging

Set LOG_LEVEL=debug for verbose output, including streamed container stderr:

# For development
LOG_LEVEL=debug pnpm run dev

# For launchd service (macOS), add to plist EnvironmentVariables:
<key>LOG_LEVEL</key>
<string>debug</string>
# For systemd service (Linux), add to unit [Service] section:
# Environment=LOG_LEVEL=debug

Debug level shows full mount configurations, the container spawn command, and streamed container stderr lines.

Inspecting Session DBs

The two session DBs are where the message flow lives. Use the in-tree query wrapper (it goes through the better-sqlite3 dep that setup already installs, avoiding a dependency on the sqlite3 CLI):

# List sessions and their agent group / messaging group from the central DB
pnpm exec tsx scripts/q.ts data/v2.db "SELECT id, agent_group_id, messaging_group_id, status, container_status, last_active FROM sessions"

# Or via the admin CLI
ncl sessions list

# Did the message reach the container? (inbound.db, host writes / container reads)
pnpm exec tsx scripts/q.ts data/v2-sessions/<group>/<session>/inbound.db \
  "SELECT seq, kind, status, timestamp FROM messages_in ORDER BY seq DESC LIMIT 10"

# Did the agent produce a reply? (outbound.db, container writes / host reads)
pnpm exec tsx scripts/q.ts data/v2-sessions/<group>/<session>/outbound.db \
  "SELECT seq, kind, timestamp FROM messages_out ORDER BY seq DESC LIMIT 10"

# Container-side processing status for each inbound message
pnpm exec tsx scripts/q.ts data/v2-sessions/<group>/<session>/outbound.db \
  "SELECT message_id, status, status_changed FROM processing_ack ORDER BY status_changed DESC LIMIT 10"

Reading the flow:

• messages_in has the message but no matching messages_out → the container never produced a reply (check processing_ack, then logs/nanoclaw.log for spawn/exit and container stderr).
• messages_out has a reply but the user never received it → a delivery problem (see issue 1 below).
• messages_in is empty → routing never reached this session (check the router log lines and the central wiring with ncl wirings list).

Repairing the gateway

If the installed credential gateway is unreachable, unhealthy, or its containers are missing, re-run its setup step from the checkout:

pnpm exec tsx setup/index.ts --step gateway

The step detects the installed gateway and checks it. For a gateway whose setup manages its own services, it also refreshes the payload and reconciles those services; otherwise it only reports whether the gateway is up, so follow that gateway's own start instructions. Don't recreate gateway containers by hand with docker run/docker rm, and keep the gateway's database volumes and keys together: never generate replacement keys for an existing database. After the step succeeds, retry the failed setup step, or restart the service (see issue 1 below) on a finished install.

Common Issues

1. "No adapter for channel type" / Messages silently lost (null platform_message_id)

Symptom: The bot stops replying. logs/nanoclaw.error.log shows repeated:

WARN No adapter for channel type channelType="telegram"
WARN No adapter for channel type channelType="signal"

The main log shows "Message delivered" entries with platformMsgId=undefined — meaning the delivery poll ran, found no adapter, and marked the message delivered without sending it.

Root cause: two NanoClaw service instances running simultaneously.

When a second service instance is active with a stale binary, it has no channel adapters registered. Its delivery poll races the working instance and wins — marking outbound messages delivered without ever sending them.

Diagnosis:

# Check for duplicate running instances
ps aux | grep 'nanoclaw/dist/index.js' | grep -v grep

# Check which services are active (Linux)
systemctl --user list-units 'nanoclaw*' --all

# Confirm channel adapters registered by the current process
grep "Channel adapter started" logs/nanoclaw.log | tail -10

Fix:

1. Identify which service has the correct binary and EnvironmentFile (the one whose log shows the expected channels — e.g. signal, telegram, cli — all started).
2. Stop and disable the stale duplicate service:

   systemctl --user stop nanoclaw.service   # or whichever is the old one
   systemctl --user disable nanoclaw.service

3. If the remaining service unit is missing EnvironmentFile, add it:

   # Edit the service unit — add this line under [Service]:
   # EnvironmentFile=/home/[user]/nanoclaw/.env
   systemctl --user daemon-reload
   systemctl --user restart nanoclaw-v2-<id>.service

4. Verify only one instance runs: ps aux | grep nanoclaw/dist/index.js | grep -v grep

Messages marked delivered with a null platform_message_id are not automatically retried. Ask the user to resend.