Opik Analytics Instrumentation

Analytics Instrumentation

Event Naming

All events MUST be prefixed with opik_. Segment routes opik_* events to PostHog. The tooling enforces this automatically, but event names defined in code should already include the prefix.

Examples: opik_onboarding_agent_name_submitted, opik_eval_suite_created, opik_optimization_created

Frontend Events

Files

• Tracking utility: apps/opik-frontend/src/lib/analytics/tracking.ts (mode-agnostic; safe to import from any project code)
• Segment init: apps/opik-frontend/src/plugins/comet/analytics/index.ts (comet-only)
• Plugin init: apps/opik-frontend/src/plugins/comet/init.tsx (comet-only)

Adding a new event

1. Add the event name to the OpikEvent const in tracking.ts:

export const OpikEvent = {
  ONBOARDING_AGENT_NAME_SUBMITTED: "opik_onboarding_agent_name_submitted",
} as const;

2. Call trackEvent from the component or hook where the action happens:

import { trackEvent, OpikEvent } from "@/lib/analytics/tracking";

trackEvent(OpikEvent.ONBOARDING_AGENT_NAME_SUBMITTED, {
  agent_name: agentName,
});

How it works

• trackEvent() safely no-ops when Segment isn't loaded (OSS mode)
• opik_ prefix is enforced at runtime as a safety net
• OPIK_ANALYTICS_ENVIRONMENT is injected into event properties automatically by trackEvent()
• Frontend custom events flow through Segment (same pipeline as backend): Segment → PostHog
• PostHog still handles automatic pageviews, user identification, and feature flags directly

Backend Events

Files

• Service: apps/opik-backend/src/main/java/com/comet/opik/infrastructure/bi/AnalyticsService.java
• Config: apps/opik-backend/src/main/java/com/comet/opik/infrastructure/AnalyticsConfig.java
• YAML config: apps/opik-backend/config.yml (under analytics:)

API

AnalyticsService exposes two overloads:

void trackEvent(String eventType, Map<String, String> properties);
void trackEvent(String eventType, Map<String, String> properties, String identity);

• 2-arg resolves identity from the current request scope via RequestContext.
• 3-arg takes an explicit identity — use it any time the call executes outside a request scope (reactive schedulers, background threads, event listeners).

How it works

• trackEvent() no-ops when OPIK_ANALYTICS_ENABLED is false (default).
• opik_ prefix is auto-prepended if missing — but keep the prefix in code for grep-ability.
• environment property is auto-injected from OPIK_ANALYTICS_ENVIRONMENT.
• Events flow: Backend → comet-stats → Segment → PostHog.
• AnalyticsService.sendEvent wraps the body in catch (RuntimeException) — callers must not add their own try/catch.

From a synchronous request handler

Inject and call inline. The 2-arg overload resolves identity from RequestContext.

private final @NonNull AnalyticsService analyticsService;

analyticsService.trackEvent("opik_onboarding_first_trace",
        Map.of("trace_id", traceId, "project_id", projectId));

From a reactive chain (doOnSuccess, doOnNext, etc.)

Two things are required: offload with Schedulers.boundedElastic() and pass identity explicitly.

Why offload: when identity is absent AnalyticsService.resolveIdentity() falls back to UsageReportService.getAnonymousId(), which is a synchronous JDBC read. Inside a doOnSuccess lambda that runs on the reactor event loop, that read blocks a scheduler-critical thread.

Why explicit identity: RequestContext is bound to the request thread via a Guice scope — inside the scheduler's lambda it throws ProvisionException, and you silently degrade to the anonymous-ID fallback, losing user attribution.

Capture userName up front from the reactor context alongside workspaceId, then pass both into the scheduled call:

return Mono.deferContextual(ctx -> {
    String workspaceId = ctx.get(RequestContext.WORKSPACE_ID);
    // Use getOrDefault on paths that internal/system callers reach without seeding USER_NAME
    // (e.g. a self-triggered cancellation written only with WORKSPACE_ID in the context).
    String userName = ctx.getOrDefault(RequestContext.USER_NAME, null);

    return someDao.write(...)
            .doOnSuccess(__ -> Schedulers.boundedElastic().schedule(
                    () -> analyticsService.trackEvent("opik_thing_happened",
                            Map.of(
                                    "thing_id", thing.id().toString(),
                                    "workspace_id", workspaceId),
                            userName)));
});

If you already depend on a Schedulers.boundedElastic().schedule(() -> { ... }) block that does other non-reactive work (e.g. a blocking datasetService.getById like ExperimentService.trackEvalSuiteRunIfApplicable), add the trackEvent call inside that existing lambda instead of nesting another.

Don'ts

• Don't add try/catch around trackEvent — sendEvent catches RuntimeException internally. Extra catches are noise and diverge from the codebase pattern.
• Don't add helper methods that only delegate to trackEvent — inline the call at the entry point. Wrap in a helper only when it encapsulates real logic (e.g. applicability check + enrichment + tracking).
• Don't re-fetch ClickHouse rows to get "fresh" values for analytics payloads — a write and a read-after-write can land on different replicas, so you may see a stale snapshot or even a spurious NotFound. Use the pre-write snapshot; some analytics drift is acceptable, a failed user-facing request is not.
• Don't add unit tests that verify(analyticsService)... — the codebase convention is for existing integration tests to exercise these paths organically. Sister analytics PRs (#6326 eval suite, #6333 onboarding, #6338 agent config) ship without emission assertions.
• Don't assume trackEvent is fully non-blocking — the Javadoc contract is aspirational; the identity-fallback path is synchronous JDBC today. Offload from reactive chains as shown above.

Python SDK Events

Files

sdks/python/src/opik/analytics/ — api.py (public surface), rules.py (when reporting is allowed), worker.py (background thread), comet_stats.py (the HTTP call). Config lives in sdks/python/src/opik/config.py, prefixed analytics_.

Identity and environment metadata are shared with Sentry error tracking, not reimplemented. Both live at the top level so neither subsystem depends on the other: opik/environment.py::get_user_identifier() (workspace name, falling back to a hostname/username hash) and opik/environment_details.py (collect_tags_once() / collect_context_once()). Analytics and error_tracking/before_send.py both read them, so an event carries the same user id, the same session_id and the same environment details as any error report from the same run. Add environment metadata there, not in either consumer.

API

One function, called explicitly as the first line of whatever is being reported:

from opik import analytics

analytics.track_event("client", "create_dataset")
analytics.track_event("integration", "openai")
analytics.track_event("evaluation", "metric_created", metric=name)

No decorators, by design: the payload is written out at the call site, so what gets sent is whatever you can read right there.

The positional arguments form a path, broadest first, and go as deep as an event needs:

analytics.track_event("integration", "bedrock")                  # the integration
analytics.track_event("integration", "bedrock", "invoke_agent")  # one part of it

The first element is a closed set (analytics.Component): client, evaluation, integration — extend it there rather than passing a new string. Every level after it is free-form, and the second is normally just the method being reported.

A longer path is a different event, not a repeat of the shorter one, so instrumenting part of a feature never silences the feature itself.

Names are composed by joining the path with a double underscore — opik_python_sdk__integration__bedrock__invoke_agent — in one private helper, so the scheme can be changed for every event at once without touching a call site. The separator is doubled so the name splits back into the path: segments are method names, so they contain single underscores but never a pair. A test enforces that (test_event_names.py); keep it true when adding events. Extra properties are keyword arguments; adding one never changes the API.

Adding an event

1. Pick the path. First element from the closed analytics.Component set — client, evaluation, integration. Second is normally the method being reported. Add further levels only to narrow a feature down ("integration", "bedrock", "invoke_agent"), remembering a longer path is a separate event, not a repeat of the shorter one.

2. Check the segments. No level may contain a double underscore, because that is the separator the name is joined with. Method names never do, so this is normally free — test_event_names.py fails the build if it is ever not.

3. Put the call on the first line of the user-facing function, before it does its work, so a call that goes on to fail still counts as usage. Do not wrap it in try/except and do not guard it with a config check; it already swallows everything and no-ops when reporting is off.

4. Decide the properties, if any. Keyword arguments, scalars only. 97 of the 98 events carry none — reach for one only when the event genuinely has variants worth splitting, as metric_created does. Never a value the user chose: report the Opik-owned name and "custom" otherwise.

5. Check it should be reported at all. Skip it if Opik calls the same entry point internally (litellm's track_completion), or if it is a per-call hot path (Opik.trace(), Opik.span(), an OTel on_start) — instrument the constructor or the user-facing function instead.

6. Verify it locally. Intercepting the HTTP call is the quickest way to see the exact payload without sending anything:

   import json, os, unittest.mock
   import httpx
   
   os.environ["OPIK_ANALYTICS_ENABLE"] = "true"
   
   sent = []
   
   def fake_post(self, url, **kwargs):
       sent.append(kwargs["json"])
       return type("R", (), {"status_code": 201})()
   
   # Patched for the block only. Replacing `post` outright leaves every later
   # request in the process - the SDK's own included - talking to the stub.
   with unittest.mock.patch.object(httpx.Client, "post", fake_post):
       from opik import analytics
   
       ...                              # exercise your new call site
       analytics.flush(timeout=10)      # batched; nothing appears without this
   
   print(json.dumps(sent, indent=2))

   Reporting is off under pytest, so this has to be a plain script, not a test. Then run the suite from the SDK directory, where it lives:

   cd sdks/python && pytest tests/unit/analytics    # 67 tests

7. Know how it will be read. The event surfaces on the Python SDK Usage dashboard, where every tile counts uniq(distinct_id). A new event needs no dashboard change to appear in the adoption tiles, which group on the name.