Migrate Required Input Dependency

Gate G — Plan confirmation

Summarize the full plan:

• Manifest changes (requires.input, policy_templates, format_version, version bump)
• Per data stream: remove legacy input: key, streams[].package, dataset:, category A/B streams[].vars only, slim template contents with {{variable}} bindings
• Variable intent matrix (categories A/B/C, Fleet visibility, template binding)
• Test and changelog changes

Ask the developer to confirm the plan before making any edits.

Phase 3 — Execute migration

Apply changes in this order (see migrate_integration_required_input_dependency.md):

1. manifest.yml — format_version, requires.input, policy_templates → package: <input>, bump version per Gate F
2. data_stream/<name>/manifest.yml — set dataset:; replace legacy input: with streams[].package; add template_path: stream.yml.hbs; declare streams[].vars for categories A/B only; remove category C vars from local manifest
3. agent/stream/stream.yml.hbs — keep only integration-owned fragments; remove all collector config merged from the input package
4. _dev/test/config.yml — policy/system requires.source if Gate E applies
5. Policy tests — test-default.yml, test-overrides.yml; generate expectations only after developer approval; confirm every Fleet-visible variable maps to the rendered agent template and user-set values take effect; confirm every Fleet-visible variable maps to the rendered agent template and user-set values take effect
6. Ingest pipelines — re-run pipeline tests; add null and missing-field cases per Gate E/F
7. System tests — extend hit assertions and traffic fixtures per Gate E
8. changelog.yml — migration (enhancement), stack constraint, field fixes (bugfix) per Gate B/F
9. Docs — elastic-package build to regenerate docs; then manual input section in _dev/build/docs/ if Gate F requires it

Do not bump unrelated packages or refactor outside migration scope.

Phase 4 — Verify

From the package directory:

elastic-package build
elastic-package check
elastic-package test -v

If system tests need variants or traffic, run what the developer confirmed in Gate E.

Verify variables in Fleet

Per the how-to guide end-to-end verification step, install the built package in a local stack and create an agent policy when possible:

1. Fleet UI ↔ template binding — every variable shown in Fleet should have a corresponding entry in the rendered agent template ({{variable}} reference or merged input-template binding). Flag any variable visible in the UI whose effective value is a hardcoded literal in stream.yml.hbs — user edits to that field will not apply.
2. User overrides take effect — change a Fleet-visible variable in the policy UI and confirm the rendered agent policy updates (policy test overrides should cover this; Fleet spot-check when a variable is not exercised in tests).
3. Defaults match intent — Fleet defaults for categories A/B match the integration manifest; category C inherited vars appear under advanced options unless explicitly redeclared.

Report:

• Build/test pass/fail with relevant log excerpts
• Policy output: data_stream.dataset and output_permissions index names (e.g. metrics-<dataset>-ep); confirm every Fleet-visible variable maps to the rendered agent template
• Fleet variable spot-check results (UI fields shown, template bindings, user override behaviour) when a local stack was available
• Dashboard spot-check on target stack when Gate F confirmed
• Platform gaps still relevant after migration:

Verification checklist

Mark each item done or N/A:

• format_version ≥ 3.6.5 and requires.input pinned to a published input version
• dataset: explicitly set on the data stream manifest when it must differ from the input default
• Local stream.yml.hbs contains only integration-owned template fragments; integration-specific or overridden values use {{variable}} references, not hardcoded literals that bypass Fleet
• Variable intent is explicit per Variable overrides: manifest vars for categories A/B, inherit input defaults when acceptable (category C) — avoid silent template hardcoding that leaves misleading values in the Fleet UI
• Variable overrides use streams[].vars, not silent template hardcoding
• Legacy input: key removed; streams[].package in place
• _dev/test/config.yml declares requires for local input package during development
• Policy tests (default + overrides): expectations confirm every Fleet-visible variable maps to the rendered agent template and user-set values take effect — review dataset, overridden defaults, sibling streams (enabled: false where required); spot-check in Fleet when policy tests do not cover a variable
• System tests pass with realistic service traffic where needed
• Pipeline regression tests for edge cases found during migration
• Changelog entries: migration, stack constraint, field-mapping fixes (use breaking-change when mapping type updates affect existing indices)
• Docs manually updated if {{ inputDocs }} is empty
• Dashboards validated on the target stack version

Decision quick-reference

Suitable input package + stack 3.6+?     → Gate 0 must pass before migrating
Legacy var differs from input default?  → B: redeclare on streams[].vars
Var only in integration template?       → A: add var + {{variable}} in slim template
Input default matches legacy?           → C: inherit; remove from local template/manifest
Input-only var on input package?        → Classify in Gate D (often C or N/A)
Per-stream vs all-streams override?     → streams[].vars vs policy_templates[].inputs[].vars
Variable intent unclear?                → Who sets it, Fleet visibility, template binding — see how-to Variable overrides
Fleet UI shows var but template ignores?→ Hardcoding anti-pattern; use manifest override or document intentional
Dataset must stay stable?               → dataset: on data stream manifest
Unpublished input package?              → _dev/test/config.yml requires.source (tests only)
Bump input pins later?                  → elastic-package requires update

Anti-patterns

• Starting migration without Gate 0 — no suitable input package or unsupported stack
• Copying patterns from packages/elastic_package_registry on main while PR #19719 is unmerged
• Migrating without confirming dataset name → silent index rename
• Skipping output_permissions index name review in policy expectations
• Hardcoding overrides in stream.yml.hbs without developer acknowledgement → Fleet UI mismatch; variable shown in UI but user edits ignored
• Using hardcoded literals in stream.yml.hbs for values that should be Fleet-configurable — use {{variable}} and manifest vars instead
• Using data_stream.dataset as a user variable when dataset: field suffices
• Leaving legacy input: alongside new streams[].package
• Running elastic-package test policy --generate and committing expectations without developer review
• Leaving full collector config in local template after switching to streams[].package
• Skipping ingest pipeline re-test after collector output shape changes
• Assuming requires.source in test config satisfies elastic-package build (build still uses registry)