Agent Workflows · By comet-ml

Run a candidate against the baseline over an Opik test suite and read the numbers back — which cases broke, which got fixed, the per-metric deltas, worst rows, and whether the two runs are…

comet-ml/opik-mcp

Surface the Opik traces worth a developer's attention, ranked by signal — Diagnostics issues first, then errors, failed tool calls, latency, regressions, and low online-eval scores.

comet-ml/opik-mcp

Add Opik tracing to an existing app and verify a real trace lands.

comet-ml/opik-mcp

Decide ship or hold for a candidate from the compare skill's numbers, against an explicit release policy — regressions, pass rate, safety-tagged cases, subgroup consistency, latency and cost…

comet-ml/opik-mcp

Root-cause a specific Opik trace, or a pattern across traces, and return a grounded explanation.

comet-ml/opik-mcp

Take a judge live on production traffic — create an Opik online evaluation rule (LLM-as-judge or Python metric) on a project with sampling, filters, variable mapping, and a cost cap, then confirm…

comet-ml/opik-mcp

Turn a failing Opik trace (or a described failure) into a repeatable regression check — a test-suite item with the trace's input and one or two binary assertions — so a fix can be verified by the…

comet-ml/opik-mcp