Devops Pipeline

DevOps Pipeline

Implement comprehensive DevOps quality gates adapted to project type, with a shift-left philosophy: run as many checks as possible locally via pre-commit so developers get fast feedback and CI is a safety net rather than the primary gate.

Core principle: if a check can run on a developer's machine, it runs there. GitHub Actions runs only what a laptop genuinely cannot — matrix version testing, secrets-dependent scans, deployment, coverage publishing — plus one cheap job proving the hooks were not bypassed.

Check Routing Table

Every check lands in exactly one lane. This table is the single source of truth: workflow steps 2 and 3, the reference files, and any config this skill generates must agree with it.

A check that fits an earlier lane must not be repeated in a later one. CI re-running the whole hook set on every push is the failure mode this skill exists to prevent.

To stay within the agent's context budget, this SKILL keeps templates short and links to references/*.md for language-specific configs, workflow templates, and the CLI E2E script.

Repo Sync Before Edits (mandatory)

Before creating/updating/deleting files in an existing repository, sync the current branch with remote:

branch="$(git rev-parse --abbrev-ref HEAD)"
git fetch origin
git pull --rebase origin "$branch"

If the working tree is not clean, stash first, sync, then restore:

git stash push -u -m "pre-sync"
branch="$(git rev-parse --abbrev-ref HEAD)"
git fetch origin && git pull --rebase origin "$branch"
git stash pop

If origin is missing, pull is unavailable, or rebase/stash conflicts occur, stop and ask the user before continuing.

Safety Rails

This skill writes config into someone else's repository and installs git hooks. Observe all of these:

• Never overwrite an existing .pre-commit-config.yaml or .github/workflows/*.yml. If the target file exists, follow this merge procedure:
  1. Copy the file to <file>.bak.
  2. Prepare the merged content. Keep user-defined hooks, jobs, and pinned rev: values.
  3. Show the user the diff between <file>.bak and the merged content.
  4. If the user confirms, write the merged file. If the user declines, leave the original file unchanged and record the merge as declined.
  5. Keep <file>.bak until the user confirms the written result.
• Show every new file before writing it. Present a new .pre-commit-config.yaml or workflow as a diff. Never write a file the user has not seen.
• Validate and dry-run before installing hooks. Step 2 runs pre-commit validate-config and pre-commit run --all-files before pre-commit install, so findings surface without blocking any commit.
• Check for an existing git hook before installing. pre-commit install preserves a foreign hook by moving it to .git/hooks/<hook>.legacy and running in migration mode. Never pass -f/--overwrite, which removes that hook silently. If the user wants the foreign hook deleted, get explicit confirmation first.
• Never run git commit --no-verify or git push --no-verify on the user's behalf. A failing hook is a finding to report, not an obstacle to route around.
• Surface failures, never suppress them. Do not add a hook id to SKIP, relax a lint rule, or lower a coverage threshold to turn a run green. Report the failure and let the user decide.
• Stop rather than guess when the stack is undetected, pre-commit is absent, or origin is missing — see Edge Cases for each.

Workflow

1. Analyze Project

Detect project characteristics:

# Check for package files and configs
ls -la package.json pyproject.toml Cargo.toml go.mod pom.xml build.gradle *.csproj 2>/dev/null
ls -la .eslintrc* .prettierrc* tsconfig.json mypy.ini setup.cfg ruff.toml 2>/dev/null
ls -la .pre-commit-config.yaml .github/workflows/*.yml 2>/dev/null

Identify:

• Languages: JS/TS, Python, Go, Rust, Java, C#, etc.
• Frameworks: React, Next.js, Django, FastAPI, etc.
• Build system: npm, yarn, pnpm, pip, poetry, cargo, go, maven, gradle
• Existing tooling: Linters, formatters, type checkers already configured
• Is this a CLI tool? — if yes, enumerate all commands/subcommands (check README, --help, click/argparse/cobra source) to build an E2E test suite

If no package manifest matches, ask the user for the language and build system before Step 2. Step 1 is done when the language, build system, existing tooling, existing config files, and CLI status (with its command list) are recorded.

2. Configure Pre-commit Hooks (maximize local coverage)

If pre-commit is absent, show the install command and stop — see Edge Cases. Do not install it automatically.

pip install pre-commit  # or brew install pre-commit

When pre-commit is available, write .pre-commit-config.yaml for the detected stack. If the file exists, use the Safety Rails merge procedure. See references/precommit-configs.md for language-specific configurations.

pre-commit stage — every commit, under 10 seconds on changed files:

• Format checks (Prettier, Black/Ruff, gofmt, rustfmt)
• Lint (ESLint, Ruff, golangci-lint, Clippy)
• Type checks (tsc, mypy)
• Security scans that work offline (Bandit, cargo-audit, gosec, detect-secrets)
• Fast unit tests
• Build/compile verification (catches import errors, compile failures early)

pre-push stage — every git push, under 60 seconds:

• Full test suite (unit + integration)
• End-to-end tests for every CLI command (see below)
• Coverage threshold enforcement
• Slower linters (full golangci-lint ruleset)

GitHub Actions only — what a laptop cannot do:

• Matrix version testing (multiple Node/Python/Go versions)
• Secrets-based scans (Snyk, SAST tools needing tokens)
• Deployment / release workflows
• Coverage upload to an external service
• Flaky or environment-sensitive tests that need a clean VM

Use the modern stage names. pre-commit 3.2 renamed commit to pre-commit and push to pre-push; the old names emit a deprecation warning on 4.x and are scheduled for removal. Always emit the new names, and run pre-commit migrate-config against any pre-existing config still using the old ones.

CLI End-to-End Testing

If the project is a CLI tool, create scripts/e2e_test.sh that exercises every command/subcommand to verify the CLI works end-to-end (not just compiles). Wire it into pre-commit on the pre-push stage.

See references/cli-e2e.md for command discovery patterns, the script template, and the pre-commit hook snippet.

Validate the config, dry-run it, then install the hooks, in this order:

1. Run pre-commit validate-config .pre-commit-config.yaml. If it exits non-zero, report the error and stop before installing any hook.
2. Run pre-commit run --all-files. This dry run executes the commit-stage hooks without installing them. Record each failing hook id as a finding.
3. Check pre-commit and pre-push in the directory git rev-parse --git-path hooks prints (not always .git/hooks). If either exists and does not contain File generated by pre-commit, tell the user it will move to <hook>.legacy.
4. Run pre-commit install.
5. Run pre-commit install --hook-type pre-push. pre-commit install alone does not install pre-push hooks.

git commit --no-verify and git push --no-verify skip every hook, and nothing local can prevent that. The CI bypass guard in step 3 is what keeps these gates enforceable — do not drop it when trimming CI.

3. Create GitHub Actions Workflows (lean CI)

If no workflow exists, write .github/workflows/ci.yml. If one exists, use the Safety Rails merge procedure, and list each existing step that duplicates a hook as a proposed removal in the diff. Remove a step only after the user confirms. Keep CI thin: the hooks already ran everything that runs locally, so CI covers the third lane of the routing table plus one guard. See references/github-actions.md for workflow templates.

CI runs exactly four kinds of thing:

1. Bypass guard — re-runs both hook stages, scoped to the pull request's diff so it costs seconds, not minutes. This is the one deliberate overlap with pre-commit, and it exists because --no-verify otherwise makes local gates optional.
2. Version matrix — the only lane that genuinely needs several runners.
3. Secrets-dependent work — scans and uploads needing tokens a laptop should not hold.
4. Deploy / release — on merge to the default branch.

The bypass guard, in full:

  hooks:
    name: Verify hooks were not bypassed
    if: github.event_name == 'pull_request'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      # ... set up the project toolchain and Python here ...
      - name: Run both hook stages over the PR diff
        run: |
          pip install pre-commit
          base="${{ github.event.pull_request.base.sha }}"
          pre-commit run --from-ref "$base" --to-ref HEAD
          pre-commit run --hook-stage pre-push --from-ref "$base" --to-ref HEAD

Four details are load-bearing; drop any one and the job fails on its first run:

• Both pre-commit run lines. The first executes commit-stage hooks only — the full suite and the CLI E2E tests live on pre-push and are silently skipped without the second. pre-commit/action@v3.0.1 shares this blind spot, so call the CLI directly.
• fetch-depth: 0. actions/checkout clones at depth 1, so the base SHA is absent and --from-ref dies on a bad object.
• if: github.event_name == 'pull_request'. On a push event github.event.before is all zeros for a branch's first push and stale after a force-push.
• The project toolchain in the same job. Hooks declared language: system shell out to npm, mypy, go, or cargo.

Keep the matrix off the hot path: gate it on push to the default branch or on a release tag, not on every PR commit. A three-version matrix on every push is triple the bill for a signal the hooks already gave locally.