Deployment Strategies

Deployment Strategies

A reference for selecting and implementing deployment strategies that minimize risk, enable zero-downtime releases, and provide fast rollback paths.

When to Activate

• Planning a deployment strategy for a new service or major release
• Implementing feature flags in an application
• Coordinating a database migration with a zero-downtime deployment
• Setting up canary releases or progressive delivery
• Defining rollback procedures for a service
• Reducing deployment risk for a high-traffic service

Strategy Comparison

Rolling Updates (Kubernetes)

Default Kubernetes behavior when you run kubectl apply. Pods are replaced incrementally — no full restart required.

strategy:
  type: RollingUpdate
  rollingUpdate:
    maxSurge: 1        # max pods above desired count during rollout
    maxUnavailable: 0  # never go below desired count (zero-downtime)

• Set maxUnavailable: 0 to guarantee zero downtime — new pods must pass readiness probes before old pods are terminated.
• Rollback: kubectl rollout undo deployment/my-service
• Target a specific revision: kubectl rollout undo deployment/my-service --to-revision=3
• Monitor progress: kubectl rollout status deployment/my-service
• Issue: slow rollback if many replicas; new version runs alongside old — both app versions must be compatible with current DB schema.

Blue/Green Deployments

Two identical environments run in parallel: Blue (live) and Green (new version). Traffic flips atomically from one to the other.

Process

1. Deploy new version to Green environment
2. Run smoke tests against Green (no user traffic yet)
3. Flip traffic: update load balancer rule or Kubernetes Service selector
4. Monitor error rate and latency for 15–30 minutes
5. Decommission Blue (or keep as instant rollback for 24 hours)

Kubernetes Implementation

Flip the Service selector to switch which deployment receives traffic.

# Blue deployment (live)
spec:
  selector:
    app: payment-service
    version: blue   # Service points here

# Green deployment (new)
spec:
  selector:
    app: payment-service
    version: green  # Update Service to point here after smoke tests

Flip command:

kubectl patch service payment-service -p '{"spec":{"selector":{"version":"green"}}}'

Considerations

• Cost: 2x infrastructure during transition window.
• Warm-up: Green must receive warming traffic (health checks, cache pre-warming) before the flip to avoid cold-start latency spikes.
• Database: Both Blue and Green versions must be compatible with the same DB schema during the transition window. Use the expand-contract pattern for migrations.

Canary Releases

Gradually shift traffic from the stable version to the new version. Automated analysis gates promotion based on SLO metrics.

• Typical progression: 5% → 25% → 50% → 100%
• Automated promotion: if error rate < 1% and p99 latency < 500 ms, advance
• Manual gate: require human approval before advancing beyond 25%
• Automated abort: if metrics breach thresholds, roll back instantly

Argo Rollouts

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: payment-service
spec:
  strategy:
    canary:
      steps:
        - setWeight: 5
        - pause: { duration: 10m }
        - setWeight: 25
        - pause: {}  # manual gate — requires human approval
        - setWeight: 50
        - pause: { duration: 10m }
        - setWeight: 100
      analysis:
        templates:
          - templateName: error-rate-check
        startingStep: 1
        args:
          - name: service-name
            value: payment-service

Promote or abort the rollout:

kubectl argo rollouts promote payment-service  # advance to next step
kubectl argo rollouts abort payment-service    # rollback to stable

Flagger (Linkerd / Istio)

Flagger integrates with service meshes for automatic traffic splitting and metric-based promotion. Define a Canary CR with analysis.metrics referencing Prometheus queries. Flagger handles weight increments and rollback automatically — no manual step definitions required.

Feature Flags

Why Feature Flags

• Decouple deployment from release: deploy code, enable for users later
• Progressive rollout: enable for 1% → 10% → 100% of users without redeploying
• Kill switch: disable instantly without a deployment or rollback
• A/B testing: different experiences for user segments based on targeting rules

Flag Lifecycle

1. Add flag (disabled by default)
2. Deploy code wrapped behind flag
3. Enable for internal users → beta users → percentage rollout → 100%
4. Remove flag and dead code (flags are technical debt — clean up within a sprint of full rollout)

Tools Comparison

Code Pattern (OpenFeature)

import { OpenFeature } from '@openfeature/server-sdk';

const client = OpenFeature.getClient();

// Simple boolean flag
const isNewCheckoutEnabled = await client.getBooleanValue(
  'new-checkout-flow',
  false,  // default value — returned if flag is missing or evaluation fails
  { targetingKey: userId }
);

if (isNewCheckoutEnabled) {
  return newCheckoutHandler(req, res);
} else {
  return legacyCheckoutHandler(req, res);
}

OpenFeature's provider abstraction means swapping from LaunchDarkly to Unleash requires changing only the registered provider — application code stays the same.

Database Migrations and Zero-Downtime Deployments

The Problem

Direct ALTER TABLE can lock tables under load. Renaming columns breaks the old app version that runs alongside the new version during a rolling deploy. Any migration that removes or renames a column must be done in phases.

Expand-Contract Pattern (Parallel Change)

Use for: adding NOT NULL columns, renaming columns or tables, changing data types.

Phase 1 — Expand (additive only):

• Add new column as NULLABLE
• Deploy application code that writes to both old and new columns
• No downtime — old app version still works with the old column

Phase 2 — Migrate:

• Backfill existing rows in batches to avoid table locks:

  UPDATE table SET new_col = old_col WHERE new_col IS NULL LIMIT 10000;

• Deploy application code that reads from the new column
• Add NOT NULL constraint once all rows are populated (now safe)

Phase 3 — Contract (remove old):

• Deploy application code that no longer references the old column
• Drop old column in a separate migration
• Can be done in a later sprint once confidence is high