Release

Data Prep Kit Release Management Skill

You are guiding the user through the data-prep-kit release process. This is a monorepo with three independently versioned packages:

All versions live in .make.versions (single source of truth). After editing versions, make set-versions propagates them to all pyproject.toml files.

NOTE — connector is independent: data-connector-lib is versioned on its own track (e.g. 0.2.x) and is not always part of a release, so the root make set-versions deliberately does not recurse into it. When the connector IS being released alongside the other packages (as it is in some releases), you must propagate its version separately: make -C data-connector-lib set-versions. After running it, verify data-connector-lib/pyproject.toml matches DPK_CONNECTOR_NEXT_RELEASE + DPK_CONNECTOR_SUFFIX. If the connector is not part of this release, leave its version untouched.

When invoked, present a menu

Ask the user which release step to perform:

1. Step 0: Dev1 Regression Build — Create a x.x.x.dev1 branch for regression testing
2. Step 1: Pending Release — Create a pending-release/x.x.x branch with clean versions
3. Step 2: Create Release — Guide creation of the release branch and tag on GitHub
4. Step 3: Post-Release Dev Setup — Bump versions and restore .dev0 suffix for new development
5. Full Release Walkthrough — Walk through all steps end-to-end
6. Release Status Check — Analyze current repo state and determine where we are in the release process

Implementation details for each step

Step 0: Dev1 Regression Build

Purpose: Create a dev1 pre-release wheel for regression testing before the final release.

1. Confirm the user is on the dev branch and it is up to date:

   git checkout dev
   git pull

2. Read .make.versions to determine the current *_NEXT_RELEASE values. Confirm the target version with the user (e.g., 1.1.8).
3. Create the dev1 branch:

   git checkout -b x.x.x.dev1

4. Edit .make.versions — set all relevant suffixes to .dev1:
   • DPK_VERSION_SUFFIX=.dev1
   • DPK_CONNECTOR_SUFFIX=.dev1
   • TRANSFORM_VERSION_SUFFIX=.dev1
5. Run make set-versions to propagate version changes to all pyproject.toml files. If the connector is part of this release, also run make -C data-connector-lib set-versions (the root recursion skips it by design).
6. Verify versions propagated correctly by reading the pyproject.toml files for each package — including data-connector-lib/pyproject.toml if the connector is being released.
7. Stage, commit, and push:

   git add .
   git commit -s -m "adding dev1 release for regression testing"
   git push --set-upstream origin x.x.x.dev1

8. Create a PR against the dev branch using gh pr create.
9. Inform the user of next steps:
   • Review and merge the PR
   • After merge, build and publish wheels for data-processing-lib and transforms. These targets live in the component Makefiles, not the repo root — run them from each component folder:

     make -C data-processing-lib build-pkg-dist publish-dist
     make -C transforms build-pkg-dist publish-dist

   • Test notebooks and confirm success before proceeding to Step 1

Step 1: Pending Release

Purpose: Remove dev suffixes and prepare the final release version.

1. Confirm the user is on the dev branch and it is up to date:

   git checkout dev
   git pull

2. Read .make.versions to determine the current version numbers. Confirm the target release version with the user.
3. Create the pending-release branch:

   git checkout -b pending-release/x.x.x

4. Edit .make.versions — clear all suffixes for the components being released:
   • DPK_VERSION_SUFFIX=
   • DPK_CONNECTOR_SUFFIX=
   • TRANSFORM_VERSION_SUFFIX=
5. Run make set-versions to propagate. If the connector is part of this release, also run make -C data-connector-lib set-versions (the root recursion skips it by design).
6. Verify versions propagated correctly, including data-connector-lib/pyproject.toml if the connector is being released.
7. Check release-notes.md for an entry for this version. If missing, prompt the user to provide release notes. Help draft them by:
   • Running git log to find commits since the last release tag
   • Categorizing changes into Transforms, General, and other sections
   • Following the existing format in release-notes.md (numbered lists with descriptions)
8. Stage, commit, and push:

   git add .
   git commit -s -m "preparing for a new release"
   git push --set-upstream origin pending-release/x.x.x

9. Create a PR against the dev branch using gh pr create.
10. Inform the user of next steps:
    • Review and merge the PR
    • After merge, build and publish wheels for data-processing-lib and transforms. These targets live in the component Makefiles, not the repo root — run them from each component folder:

      make -C data-processing-lib build-pkg-dist publish-dist
      make -C transforms build-pkg-dist publish-dist

    • Proceed to Step 2 to create the actual release