Refresh Fork

Refresh a fork

Read AGENTS.md. Rebase refreshable fork overlays onto main-next, validate the selected sources, re-pin Marin, and prepare any protected-branch promotion. An unattended run opens a draft Marin PR and names the required admin promotion; it never force-moves a stable branch. A fork_main source selector reuses an existing stable lineage and does not create a staging branch.

Refresh a group as one unit and one PR. Per-fork guidance lives beside this file: docs/vllm.md covers the vllm/tpu-inference group and the GPU release pipeline, and docs/xla.md covers the XLA PJRT fork, which is pinned outside migration.toml entirely.

In local mode, ask before pushing fork branches, opening the PR, or filing an issue. The required end-to-end test needs no extra confirmation.

Read the Descriptor

Read the target fork's section in config/external/migration.toml. It gives:

• upstream — the repo we rebase onto. Every fork has one.
• group — if present, refresh every section in the group together in one PR (read them all now); if absent, this pin refreshes alone.
• base_select — how to choose the new upstream base or existing fork source.
• pin — where the resolved pin is recorded (isolated_project uv.lock, descriptor:<path>#<section> SHA, or release:<path> prebuilt wheel); drives the re-pin step.
• branch — the maintained fork branch this pin tracks. All current forks use main. Rebase refreshes stage on main-next; fork_main selectors do not.
• e2e — the Marin end-to-end that validates the refresh.
• blocker_assignee — who owns the "can't migrate" issue.
• nuances — constraints a human must respect (torch pins, known-good ceilings).

The descriptor records how to migrate; the pin source it names holds the actual revision.

Outcome

• If no newer base is selected and no pin metadata needs repair, exit successfully with a no-op summary.
• On success, create the rollback and date tags described in docs/promotion-protocol.md for each rebased fork, then open exactly one draft PR in marin-community/marin for the fork or group after the e2e passes. A grouped refresh re-pins every group section at its staged tip in that single PR. State the exact main-next to main admin promotion still required, request the descriptor's blocker_assignee as reviewer, and monitor the PR per .agents/skills/commit/SKILL.md.
• On an unresolved blocker, do not open a PR. Create or update one marin-community/marin issue assigned to blocker_assignee, titled Fork refresh blocked: <fork> — <short reason>, with current pins, the selected base, branch names/SHAs if created, attempted fixes, the remaining failure, and artifacts.

Scratch setup

• Scratch dir: /tmp/marin-fork-refresh/<run-id> (run id: ${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT} in Actions, else a UTC timestamp plus a short label).
• Clone the fork and add its upstream remote. The fork URL is repository in the pin source (vllm/tpu.toml for descriptor pins, the [tool.uv.sources] git entry for isolated projects, the release-asset host in vllm/gpu.toml for the vllm-gpu release pin — the same marin-community/vllm repo); <upstream> is this section's upstream.

git clone <repository> <fork>
git -C <fork> remote add upstream <upstream>
git -C <fork> fetch --tags --multiple origin upstream   # --multiple fetches both remotes; without it "upstream" is read as a refspec
git -C <fork> remote set-head upstream -a                # so upstream/HEAD resolves

• Record selected bases, branch SHAs, carry/drop/fix decisions, validation, and unresolved risks for the PR.

Select the base

• base_select = upstream_main (evalchemy, harbor, MarinSkyRL, vllm-gpu): the base is the tip of the upstream default branch. These pins rebase onto upstream main; there is no release to gate on. vllm-gpu is the only unit that rebases the shared vLLM main source.
• base_select = latest_release (tpu-inference): use GitHub Releases of the fork's upstream; do not use raw tags or branches. Select the newest release where draft == false, prerelease == false, and the tag is exactly vMAJOR.MINOR.PATCH. Resolve it to a commit SHA.
• base_select = fork_main (vllm): select a full SHA already reachable from the fork's protected main, or a package-equivalent immutable release source documented by the vLLM guide. Do not rebase vLLM or consult tpu-inference's upstream LKG for this selector.

If the selected base matches the current one and no pin metadata needs repair, exit no-op. Do not walk back to older releases when the latest eligible one fails; fix the refresh or file a blocker issue.

For an isolated fork whose upstream base has not moved, there is nothing to rebase and the refresh is a no-op — even if Marin's pin lags the fork's own main. Adopting patches pushed to the fork since Marin last locked belongs to the daily external-dependency bump (ops-external-dependencies); refresh-fork runs only when there is a newer upstream base to rebase onto.

Rebase the overlay

Skip this section for base_select = fork_main: that selector reuses an exact source on the protected fork lineage.

For every rebase, branch from the selected base as main-next. This staging branch is disposable, and is distinct from protected main, which the unattended refresh leaves unchanged.

Find the base our commits currently sit on: old_base is the descriptor's upstream_base (descriptor pins) or git merge-base <fork>/main upstream/HEAD (isolated and release pins, where it is not recorded). old_tip is the head of our patches: the fork's main for isolated pins (Marin's recorded pin may lag main, so rebase from main to cover the full patch set), or the pin's stable main tip for descriptor and release pins (the descriptor's commit, or gpu.toml's source_commit). Then, onto new_base:

1. Inventory our commits in order: git log --reverse --no-merges old_base..old_tip. Merge commits (especially merges of upstream into a feature branch) are not replayed — their content comes from the new base; drop them.
2. Classify each meaningful delta: carry (still needed, not upstreamed), drop (upstream absorbed it, obsolete, or temporary), fix (intent needed, implementation must change — re-author against the current layout when upstream moved or refactored the files it touches). Before carrying anything, check whether the new base already did it: grep the base for the symbols, APIs, or dependency pins the patch introduces. Drop an absorbed backport or stale version ceiling; do not re-add duplicate or obsolete code.
3. Replay only carry and fix onto new_base in the old logical order: clean cherry-picks for carries; rewrite fixes as new commits referencing the original SHA(s). Separate genuine conflicts from cascade artifacts: a file the overlay created (absent from the new base) conflicts only because an earlier commit that added it was skipped. Classify each touched path as upstream-shared (exists at both bases) or fork-new, resolve fork-new cascades mechanically, and count conflicts only on shared files — a single abort-on-conflict pass badly overcounts and can read a tractable rebase as intractable.
4. In every retained commit body, state why it is still needed and its future drop condition. For non-obvious patches, leave a short code-adjacent rationale.
5. Run git range-diff old_base..old_tip new_base..<new_tip> as the replay audit and explain every dropped or rewritten delta in the notes and PR.
6. Audit the overlay's call-sites against the new base's API before the build. A clean cherry-pick applies textually but does not prove the upstream symbols the overlay imports or calls still exist: on a fast-moving fork a class becomes a factory function, a helper is deleted, a signature gains a required argument. Cross-check every touched constructor, signature, attribute, helper, and test against new_base before a multi-hour build. A prior vLLM refresh caught FusedMoE becoming FusedMoEFactory and removal of is_interleaved this way.
7. Keep history reviewable — no conflict artifacts, unrelated refactors, or preserved commits whose behavior is now drop. Collapse fork-infra churn (CI, workflow, or prose commits that adopt then revise then disable) to its final state rather than replaying each hop.

Stop and file a blocker when the overlay is non-linear, upstream refactored touched files that need substantial re-authoring, or many core files conflict. A fork hundreds of commits behind also needs a one-time manual catch-up outside this workflow. That catch-up replays the real commits; never reconstruct the overlay from a net diff. Diff each hand-ported fix against the fork's stable branch. Put the inventory, conflict map, and distance behind upstream in the blocker issue.