Migrate To Langfuse

Migrate to Langfuse

You are running a one-off vendor switch to Langfuse. Interview first, plan second, execute only after confirmation. Never implement from memory — fetch the current Langfuse docs listed below at execution time.

1. Interview (always, before any code)

Ask only what you cannot detect. Inspect the repo first: dependencies and imports usually reveal the source platform (langsmith, arize, phoenix, braintrust, openinference, old langfuse pins).

1. Source — which platform and version? Cloud or self-hosted? What access exists: API keys, a database connection, or only the application repo?
2. Destination — Langfuse Cloud (which region) or self-hosted? Verify credentials by presence only (e.g. [ -n "$LANGFUSE_SECRET_KEY" ]); never ask for secrets in chat, and never access .env contents or key values by any means — no shell commands that echo them, no reading .env with file tools, no grepping for values. Detect which variable names are set, nothing more.
3. Scope — which of: live tracing cutover, historical traces (including their scores, score configs, and custom model definitions), datasets, prompts, evaluators/LLM judges, experiment code. For history: everything or from a cutoff date? Ask what else exists in the source account (dashboards, alerts, annotation queues, saved views, ...). The repo cannot reveal server-side assets. Anything outside the supported list above is not covered by this skill: name it in the scope table as such, so the user plans to recreate it manually instead of discovering the gap after cutover.
4. Mode — plan only (a written runbook) or execute?

If the request contradicts the repository (it names a platform that is not present, or asks to upgrade an SDK that is not installed), stop and reconcile with the user first — never reinterpret the request as a different migration and start executing it.

Present the resulting plan (scope table plus order of operations) and get the user's confirmation before changing any file. However imperative the request sounds, it does not authorize removing or replacing the existing vendor's instrumentation before the plan is confirmed.

2. Defaults to apply

• Live cutover first. Historical data migration is optional and billed as new ingestion in Langfuse — say so before migrating history. If the source already emits OTel/OpenInference spans, add Langfuse as a second exporter on the existing instrumentation instead of re-instrumenting; the old destination is dropped only when the parallel-run window below closes.
• Datasets and prompts copy. Evaluators and judges are recreated. Experiment code is rewritten and re-run against the migrated dataset; never import old experiment scores.
• Test on a sample before the full run. Migrate ~10 traces first, link the destination traces, and wait for the user to confirm they look correct before migrating everything.
• Show progress (pages/counts) during long runs and make re-runs resumable: track sent source IDs in a state file and skip them, re-sending the same IDs creates duplicate observations (see references/reingest-history.md).
• Parallel-run window. Keep the old exporter until the user confirms the destination data, then remove it in a separate, explicit step. This binds the implementation, not just the plan: after your edits, the old vendor's exporter/instrumentation must still be present and active. Before showing a diff, re-check that no vendor import, register call, exporter endpoint, or exporter env var was deleted or repointed. Implement parallel export as one instrumentation with two exporters (both span processors on the same tracer provider) — never instrument the application twice. Duplicated generations and double-counted costs in the destination are the symptom of a second capture path.