Setup Dev

Setup Dev Environment

Interactive playbook -- follow steps in order, detect current state, skip what is already done, and ask the user when a choice is required.

Principles

• Global build cache: ccache in ~/.ccache, shared across all worktrees.
• Single clone per repo: one clone each for pg_ducklake and postgres; use git worktrees for parallel work.
• Submodule sharing: git worktree add on each submodule's git repo to share the same object store across worktrees.

Step 1: Detect state

Run these checks silently to understand what is already set up:

command -v ccache                          # compiler cache
ls pg-*/bin/pg_config 2>/dev/null          # local PG installs in workdir
ls ~/.dev/pg-*/configure 2>/dev/null       # PG source worktrees
ls "$(git rev-parse --show-toplevel)"/duckdb/Makefile 2>/dev/null  # duckdb submodule initialized?
git worktree list                          # current worktree situation

Skip any step below that is already satisfied.

Step 2: Build tools and compiler cache

Check

command -v ccache && ccache --version

If missing, ask user

Present these options using the agent's available interaction mechanism:

Also check for platform build tools:

• macOS: xcode-select -p (if missing: xcode-select --install)
• Linux: dpkg -l build-essential (if missing: sudo apt install build-essential bison flex)

Configure ccache for cmake

Check the user's shell profile:

grep CMAKE_C_COMPILER_LAUNCHER ~/.zshrc ~/.bashrc 2>/dev/null

If not configured, tell the user to add this block to ~/.zshrc or ~/.bashrc:

if command -v ccache >/dev/null; then
    export CC="ccache cc"
    export CXX="ccache c++"
    export CMAKE_C_COMPILER_LAUNCHER=ccache
    export CMAKE_CXX_COMPILER_LAUNCHER=ccache
fi

Why: duckdb and ducklake use cmake and ignore CC=ccache cc. Without CMAKE_C_COMPILER_LAUNCHER, these heavy builds (~1GB source) recompile from scratch in every worktree.

Note: if sccache is already installed (e.g. via Homebrew), cmake may pick it up automatically. That works fine -- just use sccache --show-stats instead of ccache -s to verify cache hits.

Step 3: PostgreSQL (build from source)

Always build from source, per-worktree. Uses a shared bare repo so multiple PG versions share one clone, and ccache makes rebuilds fast.

Ask the user which versions to install (usually the oldest and newest supported, e.g. 14 and 18).

One-time: clone PG source

Shared PG source lives at ~/.dev/pg-src (bare repo, ~200MB):

git clone --bare https://github.com/postgres/postgres.git ~/.dev/pg-src

For each needed version, fetch the branch and create a source worktree:

git -C ~/.dev/pg-src fetch --depth=1 origin refs/heads/REL_<VER>_STABLE:refs/heads/REL_<VER>_STABLE
git -C ~/.dev/pg-src worktree add ~/.dev/pg-<VER> REL_<VER>_STABLE

Build and install into current workdir

WT=$(git rev-parse --show-toplevel)
NCPU=$(nproc 2>/dev/null || sysctl -n hw.ncpu)

for VER in 14 18; do
    pushd ~/.dev/pg-$VER
    ./configure --prefix="$WT/pg-$VER" \
        --without-icu --without-readline --without-libxml
    make -j"$NCPU" && make install
    popd
done

PG_CONFIG will be: $(pwd)/pg-<VER>/bin/pg_config

Add pg-*/ to .git/info/exclude to keep git status clean:

exclude_file=$(git rev-parse --git-dir)/info/exclude
grep -qF 'pg-*/' "$exclude_file" 2>/dev/null || echo 'pg-*/' >> "$exclude_file"

Time: ~15-20 min per version on first run, ~1-2 min on reruns (ccache).

Step 4: Submodules

duckdb (submodule)

pg_ducklake lives in the libpgduckdb monorepo: the libpgddb kernel is plain in-repo source at libpgduckdb/ (repo root) -- nothing to initialize. The DuckDB source is a submodule at the repo root, duckdb/. Initialize it:

git -C "$(git rev-parse --show-toplevel)" submodule update --init --depth=1 duckdb

Time: 5-10 min on first run (duckdb is large even with shallow clone).

ducklake (submodule + patches)

third_party/ducklake is a git submodule pinned to a community duckdb/ducklake commit. Our divergence from community lives as an ordered series of patch files third_party/ducklake-NNN-<desc>.patch, applied onto the pristine checkout at build time (apply-once, stamp-guarded -- see the DUCKLAKE_STAMP rule in Makefile). The build inits the submodule non-recursively (its inner duckdb / extension-ci-tools submodules are only for ducklake's own CI and are not needed) and applies the patches in numeric order; make clean drops the stamp so the next build re-applies from a clean reset. The submodule working tree is therefore intentionally left dirty (patched); the gitlink stays at the pinned community commit.

To inspect our customization, read the ducklake-NNN-*.patch files. To bump the community base: repoint the submodule gitlink, regenerate/rebase the patches against the new commit, and re-verify (make ... && make check-regression).