Oma Observability

Observability Agent - Intent-based Router

Scheduling

Goal

Route, design, tune, and review observability work across MELT+P signals, layers, boundaries, vendor categories, transport choices, meta-observability, and incident forensics.

Intent signature

• User asks for observability, telemetry, OTel, metrics, logs, traces, profiles, SLOs, RUM, APM, incident forensics, trace propagation, transport tuning, or observability-as-code.
• User needs vendor/category routing or observability architecture instead of a single vendor's already-covered setup.

When to use

• Setting up an observability pipeline (OTel SDK + Collector + vendor backend)
• Designing traceability across service and domain boundaries (W3C propagators, baggage, multi-tenant, multi-cloud)
• Tuning transport layer (UDP/MTU, OTLP gRPC vs HTTP, Collector DaemonSet vs sidecar topology)
• Running incident forensics (6-dimension localization: code / service / layer / host / region / infra)
• Selecting a vendor category (OSS full-stack vs commercial SaaS vs high-cardinality specialist vs profiling specialist)
• Implementing observability-as-code (Grafana Jsonnet dashboards, PrometheusRule CRD, OpenSLO YAML, SLO burn-rate alerts)
• Meta-observability (pipeline self-health, clock skew detection, cardinality guardrails, retention matrix)
• Covering the MELT+P signal set: metrics, logs, traces, profiles (OTEP 0239), cost (OpenCost), audit (SOC2/ISO), privacy (GDPR/PIPA)
• Migrating off deprecated tools (Fluentd → Fluent Bit or OTel Collector, per CNCF 2025-10 guide)

When NOT to use

• LLM ops (prompt versioning, evals, gen_ai span deep dive) — use Langfuse, Arize Phoenix, LangSmith, or Braintrust directly
• Data pipeline lineage — use OpenLineage + Marquez, dbt test, or Airflow lineage backends
• IoT / hardware / datacenter physical-layer telemetry (IPMI, BMC, SNMP) — use vendor DCIM tooling (Nlyte, Sunbird, Device42)
• Chaos engineering orchestration — use Chaos Mesh, Litmus, Gremlin, or ChaosToolkit (this skill consumes their telemetry; it does not orchestrate chaos)
• GPU / TPU infrastructure observability — use NVIDIA DCGM Exporter + Prometheus
• Software supply chain (SBOM, attestation) — use sigstore (cosign / rekor), in-toto framework, SLSA level attestations
• Incident response workflow (on-call rotation, paging, escalation) — use PagerDuty, OpsGenie, or Grafana OnCall
• Single-vendor setup already fully covered by that vendor's own published skill — invoke the vendor skill directly

Expected inputs

• Observability intent, target system, architecture boundary, signals, vendor context, and incident symptoms if any
• Existing OTel/collector/vendor configs, dashboards, SLOs, trace/log/metric examples, or deployment topology

Expected outputs

• Routed observability guidance, setup/migration/tuning plan, incident-forensics path, alerting/SLO guidance, or observability-as-code recommendations
• Transport, meta-observability, privacy, audit, and retention checks
• Vendor delegation target when appropriate

Dependencies

• OTel/W3C/CNCF references and resources under resources/
• Vendor categories, matrix, standards, incident forensics, meta-observability, transport, layers, boundaries, and signal guides

Control-flow features

• Branches by intent, vendor category, layer/boundary/signal matrix, transport topology, privacy/audit risk, and incident localization dimension
• May read/write observability config and docs; generally delegates vendor-specific implementation
• Requires live status verification for load-bearing CNCF/vendor currency

Structural Flow

Entry

1. Classify the intent: setup, migrate, investigate, alert, trace, tune, or route.
2. Identify layers, boundaries, signals, and vendor category.
3. Load only the relevant resource guide(s).

Scenes

1. PREPARE: Classify intent and matrix coverage.
2. ACQUIRE: Read configs, topology, telemetry examples, or incident signals.
3. REASON: Route vendor/category, tune transport, assess meta-observability, or localize incident.
4. ACT: Produce setup/migration/tuning/alert/trace/forensics guidance or config changes.
5. VERIFY: Check pipeline health, clock skew, cardinality, retention, privacy, and audit concerns.
6. FINALIZE: Report route, evidence, risks, and handoff references.

Transitions

• If a vendor-owned skill fully covers setup, delegate instead of duplicating docs.
• If Fluentd appears, recommend Fluent Bit or OTel Collector migration.
• If incident investigation is requested, use 6-dimensional localization.
• If transport tuning appears, load transport-specific resources.

Failure and recovery

• If live CNCF/vendor status is load-bearing, verify current status.
• If telemetry samples are missing, provide instrumentation/collection steps before analysis.
• If scope belongs to out-of-scope domains, route to external authoritative tools.

Exit

• Success: observability path is routed, evidence-backed, and checks are explicit.
• Partial success: missing telemetry, stale vendor status, or external-domain handoff is explicit.

Logical Operations

Actions

Tools and instruments

• OTel/CNCF/W3C standards references
• Vendor categories, matrix, incident forensics, meta-observability, transport and signal guides
• Optional CLI/config tooling from the target stack

Canonical workflow path

1. Classify intent: setup, migrate, investigate, alert, trace, tune, or route.
2. Select layer/boundary/signal coverage from `resources/matrix.md`.
3. Load the specific vendor, transport, incident, or signal guide before producing guidance.

When CNCF/vendor status is load-bearing, verify live state at https://landscape.cncf.io.

Resource scope

Preconditions

• Observability intent and system boundary are identifiable.
• Relevant telemetry/config evidence is available or missing evidence is stated.

Effects and side effects

• May recommend or modify observability config, dashboards, alerts, and instrumentation docs.
• May route to vendor-owned skills or external tools.

Guardrails

1. Classify intent before routing: every query goes through intent classification — setup | migrate | investigate | alert | trace | tune | route
2. Category-first, not vendor-registry: delegate to vendor-owned skills via resources/vendor-categories.md; do not duplicate their documentation
3. Transport tuning is the moat: UDP/MTU thresholds, OTLP protocol selection, Collector topology, and sampling recipes are in-skill depth that other skills do not cover
4. Meta-observability is non-negotiable: always validate pipeline self-health, clock sync (< 100 ms drift), cardinality, and retention before declaring setup complete
5. CNCF-first preference: Prometheus, Jaeger, Thanos, Fluent Bit, OpenFeature (Graduated 2024-11), Flagger, Falco (Graduated); OpenTelemetry, Cortex, OpenCost (Incubating)
6. Fluentd is deprecated: per CNCF 2025-10 migration guide, recommend Fluent Bit or OTel Collector for all new and migration work
7. W3C Trace Context as default propagator: translate per cloud (AWS X-Ray X-Amzn-Trace-Id, GCP Cloud Trace, Datadog, Cloudflare, Linkerd) via boundaries/cross-application.md
8. Privacy before features: PII redaction, sampling-aware baggage rules, and compliance (SOC2/ISO immutable audit + GDPR/PIPA erasure) are applied at collection, not only at storage
9. Domain-level trust: all vendor and tool references are timestamped as of 2026-Q2; verify live status at https://landscape.cncf.io
10. No stub in final deliverable: scaffolds are editing anchors only during build phase; remove before output

Out of Scope (use external tools)

The combinations below are outside this skill's boundary. The external tools listed are authoritative for each domain.