Cortext Self Diagnosis

Cortext Self-Diagnosis

Debugging the framework you are running inside. This skill covers cortextOS infrastructure: PTY sessions, the daemon, the message bus, state markers, crons, and the Telegram path — through to an upstream fix when the cause is real.

Not this skill: stale tasks, stale goals, overdue human tasks, fleet workload health. That is system-diagnostics. The dividing line is simple — if the machinery is misbehaving, you are here; if the machinery works and the work is stuck, you are there.

The one thing that makes this different

You are debugging the system that is currently executing you. Your session is a PTY child of the daemon you are about to inspect. This has three consequences that will bite you if you forget them:

Write evidence to disk before you touch anything that restarts. A daemon restart kills and respawns your own PTY. Anything you know only in context is gone. The evidence bundle exists so your findings survive your own restart.

Never restart the daemon or your own agent without saying so first. From the user's side that looks identical to the bug — the agent goes quiet. Tell them what you are about to do, that you may drop mid-sentence, and roughly when you will be back.

You are a witness, and witnesses have blind spots. If you are the agent that is misbehaving, your account of yourself is the least reliable evidence you have. Prefer the logs to your own memory, and when the symptom is about you, say so plainly to the user and lean harder on the files.

Stance: evidence before repair

The failure mode this skill exists to prevent is the confident guess — changing a config, restarting something, declaring it fixed, and never learning what happened. It usually "works" because restarts paper over transient state, and the bug returns next week with the trail gone cold.

So: find the artifact that proves it before you touch anything. A hypothesis you cannot point at a log line for is a hunch. Say "I think" out loud when it is a hunch — users make much better decisions when they know which of your claims are load-bearing.

If the evidence genuinely does not resolve it, that is a real outcome. Say what you checked, what you ruled out, and what would settle it. That beats a fix that might be unrelated.

Phase 0 — Orient

Two minutes, before any commands.

Get the symptom concrete. "Cortext is broken" is not actionable; "opsbot stopped replying on Telegram around 2pm, still nothing" is. You need:

• Which agent(s) — one, several, or the whole fleet
• What you expected vs. what happened
• When it started, even roughly, and whether it is ongoing or over
• What changed recently — an update, a config edit, a new agent, a reboot, a rebuild. This single question resolves a large share of cases.

Then resolve the runtime root, because everything else hangs off it:

# Precedence: explicit env, then per-agent env file, then instance default,
# then the legacy root some installs still use.
echo "${CTX_ROOT:-$(grep -h '^CTX_ROOT=' ~/.cortextos-env */.cortextos-env 2>/dev/null | head -1 | cut -d= -f2)}"
ls -d ~/.cortextos/default ~/.business-os 2>/dev/null

Layout under the root (references/surface-map.md has the full map):

logs/{agent}/       stdout.log stderr.log restarts.log crashes.log
                    fast-checker.log activity.log .crash_count_today
state/{agent}/      heartbeat.json + .onboarded .force-refresh .handoff markers
inbox|inflight|processed|outbox/{agent}/    message lifecycle
config/             enabled-agents.json
orgs/{org}/         tasks/ approvals/ analytics/ crons.json

Daemon output lives outside the root, under PM2: ~/.pm2/logs/cortextos-daemon-out.log and -error.log.

Phase 1 — Collect evidence

Run the collector. It snapshots every surface at once into a timestamped bundle, so you are reading a consistent moment rather than files that shift under you:

bash scripts/collect_evidence.sh --agent <name> --since 2h
# whole fleet: omit --agent

It prints a bundle path. Read it there.

The script is a fast path, not the method. It knows the standard layout; it does not know your install's quirks, and it cannot tell which of 400 log lines matters. Installs drift — legacy roots, relocated logs, custom instances. When the script comes back thin or empty, that is a signal to go look by hand, not a verdict that nothing is wrong. references/surface-map.md documents each surface so you can read any of them directly:

tail -200 "$ROOT/logs/<agent>/stdout.log"
tail -50  "$ROOT/logs/<agent>/restarts.log"
tail -100 ~/.pm2/logs/cortextos-daemon-error.log

Also run the framework's own preflight — it catches environment breakage (Node version, node-pty native module, PM2, CLI auth) faster than reading logs:

cortextos doctor

Phase 2 — Read the right surface first

Symptom routes you to the surface most likely to hold the answer. Start there, then widen. Full playbooks with confirm/refute criteria are in references/symptom-playbooks.md — read it when your symptom is on this list, as each playbook carries the specific log lines that distinguish similar-looking causes.

That last row matters more than it looks. A large share of "bugs" are the agent faithfully following an instruction someone wrote. Before you go hunting in the daemon, read what the agent was actually told. These systems are suggestible and a stray line in a bootstrap file changes behavior a lot.

When cortext's own logs are clean but behavior is still wrong, the fault is often one layer down in the harness — Claude Code, Codex, or OpenCode itself, or an API error. Those surface in the session transcript, not in cortext's logs. That is the place to look when everything cortext-side appears healthy.

Phase 3 — Hypothesis, then confirmation

State the hypothesis in one sentence, in causal form: the fast-checker stopped polling at 14:02, so messages queued in inbox and never reached the PTY.

Then go find the artifact that would be true if you are right — and, just as important, the one that would be true if you are wrong. Actively look for the second one. Confirmation bias is the main way debugging goes sideways: the first plausible story absorbs every subsequent observation.

Timestamps are your strongest tool. Line up the moment the symptom started against restarts, crashes, daemon events, and message timestamps. Causes precede effects, which sounds obvious and eliminates suspects fast.

Timing trap worth knowing: after any daemon restart there is a burst of startup noise — reconnects, re-injections, heartbeat churn — that looks a lot like the original symptom. Give it ten minutes before judging whether a restart fixed anything. Calling it too early, in either direction, is one of the easier ways to waste an hour.