Sensing Track

Sensing Event History

Quick Start

The primary data source is the flow events JSONL at /root/local/flow_events_YYYY-MM-DD.jsonl. Each file covers one calendar day (7-day retention, no size-rotation mid-day). Use Bash + jq to query it.

Important: Always use the absolute path /root/local/ — the read tool cannot access files outside the workspace, so use exec (Bash) for all JSONL queries.

Persistent camera snapshots are stored under /var/lib/hal/snapshots/sensing_<prefix>/<ms>.jpg (72h TTL, 50 MB cap) — one subdir per event kind:

Reference these when the user asks what happened visually.

JSONL format

Each line is a JSON object:

{"kind":"enter","node":"sensing_input","ts":1712345678.123,"seq":42,"trace_id":"run-abc","data":{"type":"presence.enter","message":"Person detected — new: friend (gray); faces in frame: 1 (gray)\n[snapshot: /var/lib/hal/snapshots/sensing_face/1712345678123.jpg]"},"version":"1.2.3"}
{"kind":"exit","node":"sensing_input","ts":1712345678.456,"seq":43,"trace_id":"run-abc","duration_ms":332,"data":{"path":"agent","run_id":"run-abc"},"version":"1.2.3"}

Key fields:

• node — filter on "sensing_input" for sensing events
• kind — "enter" = event received, "exit" = event processed (with duration_ms)
• data.type — event type: presence.enter, presence.leave, motion, motion.activity, sound, light.level, voice, voice_command, emotion.detected, speech_emotion.detected
• data.message — natural-language description; may contain [snapshot: /var/lib/hal/snapshots/sensing_<prefix>/<ms>.jpg]
• data.path — in exit records: "agent" (forwarded), "local" (handled locally), or has "error" key (failed/dropped)
• ts — Unix timestamp (seconds with fractional ms)
• trace_id — correlates enter/exit and links to agent turn

Tools

Bash — jq, cat, date arithmetic. No writes.

Query recipes

Timezone — always set before date arithmetic

export TZ=$(cat /etc/timezone)

All sensing events in a time range

export TZ=$(cat /etc/timezone)
DATE="$(date +%Y-%m-%d)"
FROM_TS=$(date -d "$DATE 22:00:00" +%s)
TO_TS=$(date -d "$DATE 23:59:59" +%s)
jq -c 'select(.node=="sensing_input" and .kind=="enter" and .ts >= '"$FROM_TS"' and .ts <= '"$TO_TS"')' \
  "/root/local/flow_events_${DATE}.jsonl"

Events of a specific type in the last N hours

Use "motion" for raw motion, "motion.activity" for activity analysis (HAL-categorised — bucket names drink/break/celebrate and raw Kinetics sedentary labels like using computer, writing, reading). Most queries want both:

For any relative range, select every local calendar day's file intersecting that range, including yesterday when crossing midnight. Run this range setup and the chosen query in the same Bash call; do not assume variables survive between tool calls. GNU date -d is available on the device.

export TZ=$(cat /etc/timezone)
SINCE=$(date -d "1 hour ago" +%s)
UNTIL=$(date +%s)
first_day=$(date -d "@$SINCE" +%F)
last_day=$(date -d "@$UNTIL" +%F)
range_files=()
range_day=$first_day
while [[ "$range_day" < "$last_day" || "$range_day" == "$last_day" ]]; do
  range_file="/root/local/flow_events_${range_day}.jsonl"
  if [ -r "$range_file" ]; then
    range_files+=("$range_file")
  else
    printf 'History unavailable for %s\n' "$range_day" >&2
  fi
  range_day=$(date -d "$range_day +1 day" +%F)
done
if [ "${#range_files[@]}" -gt 0 ]; then
  jq -c --argjson since "$SINCE" --argjson until "$UNTIL" \
    'select(.node=="sensing_input" and .kind=="enter" and .ts >= $since and .ts <= $until and (.data.type=="motion" or .data.type=="motion.activity"))' \
    "${range_files[@]}"
fi

Any activity in the last N minutes

Use the same range setup above with SINCE=$(date -d "30 minutes ago" +%s) and remove the type predicate from the jq filter. Keep both time bounds and all range_files; even 30 minutes can span two local dates.

Presence events only (who came by)

Names in messages are lowercase (friend (gray)). Use test() with "i" flag for case-insensitive search:

TODAY=$(date +%Y-%m-%d)
# All presence events
jq -c 'select(.node=="sensing_input" and .kind=="enter" and (.data.type=="presence.enter" or .data.type=="presence.leave"))' \
  "/root/local/flow_events_${TODAY}.jsonl"
# Search for a specific person (case-insensitive)
jq -c 'select(.node=="sensing_input" and .kind=="enter" and .data.type=="presence.enter" and (.data.message | test("gray";"i")))' \
  "/root/local/flow_events_${TODAY}.jsonl"

Events spanning multiple days

export TZ=$(cat /etc/timezone)
YESTERDAY=$(date -d "yesterday" +%Y-%m-%d)
TODAY=$(date +%Y-%m-%d)
cat "/root/local/flow_events_${YESTERDAY}.jsonl" "/root/local/flow_events_${TODAY}.jsonl" \
  | jq -c 'select(.node=="sensing_input" and .kind=="enter" and .ts >= '"$FROM_TS"' and .ts <= '"$TO_TS"')'

Dropped events (agent was busy)

TODAY=$(date +%Y-%m-%d)
jq -c 'select(.node=="sensing_input" and .kind=="exit" and .data.error != null)' \
  "/root/local/flow_events_${TODAY}.jsonl"

List snapshots (72h TTL — older files may be purged)

Snapshots are bucketed into sensing_face/ (presence), sensing_motion_activity/, sensing_emotion/. Recurse into subdirs:

# Most-recent snapshots across all categories
find /var/lib/hal/snapshots -type f -name '*.jpg' -printf '%T@ %p\n' | sort -rn | head -20 | cut -d' ' -f2-

# Only a specific category
ls -lt /var/lib/hal/snapshots/sensing_motion_activity/ | head -20

Pose buckets (posture history)

Posture snapshots are NOT in /var/lib/hal/snapshots/ — they live in tmp under a per-window bucket layout at /tmp/hal-sensing-snapshots/sensing_pose/buckets/<bucket_id>/. A bucket only exists when a tumbling window closed with bad posture (bad_ratio >= POSE_BAD_RATIO). Kept buckets survive ~2 days (POSE_BUCKET_KEEP_S); windows that didn't fire a nudge are deleted immediately, so the buckets you can see are by definition "bad posture" sessions.

Each kept bucket contains:

• <sample_ts>_<score>.jpg — annotated frame per sample (skeleton overlay + RULA score)
• bucket.json — manifest:
  • bucket_id, window_start_ts, window_end_ts, kept: true
  • summary — same shape as the [posture_summary:] block on motion.activity (bad_ratio, dominant_region, samples, …)
  • samples[] — {ts, score, risk_level, filename, left, right} (per-side RULA body_scores + angles)
  • worst_snapshots[] — pre-selected worst filenames (the ones the device auto-attaches to /dm on posture nudges)

# List recent buckets (newest first)
ls -lt /tmp/hal-sensing-snapshots/sensing_pose/buckets/ | head -10

# Read a specific bucket's manifest
jq . /tmp/hal-sensing-snapshots/sensing_pose/buckets/1779259742/bucket.json

# Buckets that closed in the last 2 hours
find /tmp/hal-sensing-snapshots/sensing_pose/buckets -maxdepth 1 -type d -mmin -120 -name '[0-9]*' | sort

# Worst-frame paths from the latest kept bucket
LATEST=$(ls -t /tmp/hal-sensing-snapshots/sensing_pose/buckets/ | head -1)
jq -r '.worst_snapshots[]' "/tmp/hal-sensing-snapshots/sensing_pose/buckets/${LATEST}/bucket.json" \
  | sed "s|^|/tmp/hal-sensing-snapshots/sensing_pose/buckets/${LATEST}/|"

# Today's bad-posture sessions — bucket id == window_start unix-seconds
TODAY_START=$(date -d "today 00:00" +%s)
for b in /tmp/hal-sensing-snapshots/sensing_pose/buckets/*/bucket.json; do
  jq --arg start "$TODAY_START" 'select((.window_start_ts | floor) >= ($start | tonumber)) | {bucket_id, dominant: .summary.dominant_region, bad_ratio: .summary.bad_ratio, started: .window_start_ts}' "$b"
done

Note: motion.activity event messages contain [pose_bucket: <id>] and [pose_worst: <fn1>,<fn2>,...] markers — parse these out of data.message when you need to map a sensing_input record to its bucket. Markers are present whenever a posture nudge folded in.

Fallback: system log

For detailed debugging or when you need Go-side log context (errors, warnings, lifecycle details), fall back to ${OS_LOG:-/var/log/os-server.log}:

LOG="${OS_LOG:-/var/log/os-server.log}"
sed 's/\x1b\[[0-9;]*m//g' "$LOG" | grep "sensing event received"

The system log uses lumberjack rotation (1 MB cap, 3 backups) — it may miss data during high traffic. Use it only when JSONL doesn't have enough detail, or when investigating bugs.

Mood history

A dedicated mood history log tracks user mood per user. Only the user's emotional state is logged — not system events or device emotions. Each user's mood data lives in their own directory.

Read API:

# Current user's mood history (auto-detects who's present)
curl -s "http://127.0.0.1:5000/api/openclaw/mood-history?date=$(date +%Y-%m-%d)&last=100"

# Specific user's mood history
curl -s "http://127.0.0.1:5000/api/openclaw/mood-history?user=gray&date=$(date +%Y-%m-%d)&last=100"

Write: Follow the Mood skill to log user mood from camera or conversation.

{"ts":1776138500,"seq":1,"hour":10,"mood":"happy","source":"camera","trigger":"laughing"}
{"ts":1776139200,"seq":2,"hour":10,"mood":"stressed","source":"conversation","trigger":"user said feeling overwhelmed"}

Storage: /root/local/users/{name}/mood/YYYY-MM-DD.jsonl (30-day retention).

Device sleep history

Your own sleep — when you went to sleep and woke up, not the user's. Use it for "how many times have you slept?", "when do you usually go to sleep?", "did you sleep while I was out?".

Storage: /root/local/device/sleep/YYYY-MM-DD.jsonl (30-day retention). One line per transition, appended:

{"ts":1758000000.12,"local":"2026-09-16T22:00:00+07:00","tz":"Asia/Ho_Chi_Minh","date":"2026-09-16","hour":22,"event":"sleep","emotion":"sleepy","source":"api"}
{"ts":1758021600.45,"local":"2026-09-17T06:00:00+07:00","tz":"Asia/Ho_Chi_Minh","date":"2026-09-17","hour":6,"event":"wake","emotion":"stretching","source":"button"}

• local — the device's own wall-clock with its UTC offset. Say times to the user from this field, never by converting ts yourself.
• tz — the zone that offset came from. Empty means the device could not resolve its zone and the time is naive: report it as approximate rather than quoting it exactly.
• ts — epoch seconds. Use it for ordering and for durations; it is the only field safe to subtract, because the user can change the zone between two rows.
• event — sleep or wake. Count sleep rows; a sleepy re-sent to an already sleeping device writes nothing, so every row is a real transition.
• source — who caused it: button / touch / MPR121 (someone did it by hand), api (your own [HW:/emotion:sleepy] marker, or the web UI).
• emotion — sleepy going in; stretching or greeting coming out.

export TZ=$(cat /etc/timezone)
# Times slept today
jq -c 'select(.event=="sleep")' "/root/local/device/sleep/$(date +%Y-%m-%d).jsonl" | wc -l

# Times slept over everything retained
cat /root/local/device/sleep/*.jsonl | jq -c 'select(.event=="sleep")' | wc -l

# Usual bedtime hour
cat /root/local/device/sleep/*.jsonl \
  | jq -r 'select(.event=="sleep") | .hour' | sort -n | uniq -c | sort -rn | head -3

# Recent transitions with readable local times
cat /root/local/device/sleep/*.jsonl | jq -c '{local,event,source}' | tail -6

# How long each sleep lasted — pair every sleep with the wake that follows it.
# Durations come from `ts`, the times shown come from `local`.
cat /root/local/device/sleep/*.jsonl | jq -s -r '
  def shown: .local // ((.ts | strftime("%Y-%m-%dT%H:%M:%SZ")) + " (UTC)");
  sort_by(.ts)
  | reduce .[] as $r ({open:null, out:[]};
      if $r.event == "sleep" then .open = $r
      elif .open then .out += [{from: (.open | shown), to: ($r | shown),
                                mins: (($r.ts - .open.ts) / 60 | floor)}] | .open = null
      else . end)
  | .out[] | "\(.from) → \(.to)  (\(.mins) min)"'

Two gaps to report honestly rather than paper over:

• A trailing sleep with no wake after it means the device is still asleep, or was powered off while sleeping. The pairing drops it instead of inventing an end.
• Rows written before local existed have no local time. The shown fallback above prints those in UTC, labelled — do not present a UTC time as the user's local time.

Do not confuse this with action:"sleep" in the wellbeing log — that is the user telling you they are going to bed (see the Habit skill). This file is about your own body.

This file is the only complete record. Flow events carry hw_emotion rows for sleeps your own marker fired, but the button never reaches os-server, so counting from flow_events_*.jsonl silently undercounts and misses exactly the times a person put you to sleep by hand. Count from here.

Nothing before the journal existed can be recovered — it was never written down anywhere. If the files start mid-window, say how far back you can actually see rather than reporting a total as if it were lifetime.