Debugging Techniques

Debugging Techniques

Purpose

Provides systematic debugging workflows for local, remote, container, and production environments across Python, Go, Rust, and Node.js. Covers interactive debuggers, container debugging with ephemeral containers, and production-safe techniques using correlation IDs and distributed tracing.

When to Use This Skill

Trigger this skill for:

• Setting breakpoints in Python, Go, Rust, or Node.js code
• Debugging running containers or Kubernetes pods
• Setting up remote debugging connections
• Safely debugging production issues
• Inspecting goroutines, threads, or async tasks
• Analyzing core dumps or stack traces
• Choosing the right debugging tool for a scenario

Quick Reference by Language

Python Debugging

Built-in: pdb

# Python 3.7+
def buggy_function(x, y):
    breakpoint()  # Stops execution here
    return x / y

# Older Python
import pdb
pdb.set_trace()

Essential pdb commands:

• list (l) - Show code around current line
• next (n) - Execute current line, step over functions
• step (s) - Execute current line, step into functions
• continue (c) - Continue until next breakpoint
• print var (p) - Print variable value
• where (w) - Show stack trace
• quit (q) - Exit debugger

Enhanced tools:

• ipdb - Enhanced pdb with tab completion, syntax highlighting (pip install ipdb)
• pudb - Terminal GUI debugger (pip install pudb)
• debugpy - VS Code integration (included in Python extension)

Debugging tests:

pytest --pdb  # Drop into debugger on test failure

For detailed Python debugging patterns, see references/python-debugging.md.

Go Debugging

Delve - Official Go debugger

Installation:

go install github.com/go-delve/delve/cmd/dlv@latest

Basic usage:

dlv debug main.go              # Debug main package
dlv test github.com/me/pkg     # Debug test suite
dlv attach <pid>               # Attach to running process
dlv debug -- --config prod.yaml  # Pass arguments

Essential commands:

• break main.main (b) - Set breakpoint at function
• break file.go:10 (b) - Set breakpoint at line
• continue (c) - Continue execution
• next (n) - Step over
• step (s) - Step into
• print x (p) - Print variable
• goroutine (gr) - Show current goroutine
• goroutines (grs) - List all goroutines
• goroutines -t - Show goroutine stacktraces
• stack (bt) - Show stack trace

Goroutine debugging:

(dlv) goroutines                 # List all goroutines
(dlv) goroutines -t              # Show stacktraces
(dlv) goroutines -with user      # Filter user goroutines
(dlv) goroutine 5                # Switch to goroutine 5

For detailed Go debugging patterns, see references/go-debugging.md.

Rust Debugging

LLDB - Default Rust debugger

Compilation:

cargo build  # Debug build includes symbols by default

Usage:

rust-lldb target/debug/myapp   # LLDB wrapper for Rust
rust-gdb target/debug/myapp    # GDB wrapper (alternative)

Essential LLDB commands:

• breakpoint set -f main.rs -l 10 - Set breakpoint at line
• breakpoint set -n main - Set breakpoint at function
• run (r) - Start program
• continue (c) - Continue execution
• next (n) - Step over
• step (s) - Step into
• print variable (p) - Print variable
• frame variable (fr v) - Show local variables
• backtrace (bt) - Show stack trace
• thread list - List all threads

VS Code integration:

• Install CodeLLDB extension (vadimcn.vscode-lldb)
• Configure launch.json for Rust projects

For detailed Rust debugging patterns, see references/rust-debugging.md.

Node.js Debugging

Built-in: node --inspect

Basic usage:

node --inspect-brk app.js       # Start and pause immediately
node --inspect app.js           # Start and run
node --inspect=0.0.0.0:9229 app.js  # Specify host/port

Chrome DevTools:

1. Open chrome://inspect
2. Click "Open dedicated DevTools for Node"
3. Set breakpoints, inspect variables

VS Code integration: Configure launch.json:

{
  "type": "node",
  "request": "launch",
  "name": "Launch Program",
  "program": "${workspaceFolder}/app.js"
}

Docker debugging:

EXPOSE 9229
CMD ["node", "--inspect=0.0.0.0:9229", "app.js"]

For detailed Node.js debugging patterns, see references/nodejs-debugging.md.

Container & Kubernetes Debugging

kubectl debug with Ephemeral Containers

When to use:

• Container has crashed (kubectl exec won't work)
• Using distroless/minimal image (no shell, no tools)
• Need debugging tools without rebuilding image
• Debugging network issues

Basic usage:

# Add ephemeral debugging container
kubectl debug -it <pod-name> --image=nicolaka/netshoot

# Share process namespace (see other container processes)
kubectl debug -it <pod-name> --image=busybox --share-processes

# Target specific container
kubectl debug -it <pod-name> --image=busybox --target=app

Recommended debugging images:

• nicolaka/netshoot (~380MB) - Network debugging (curl, dig, tcpdump, netstat)
• busybox (~1MB) - Minimal shell and utilities
• alpine (~5MB) - Lightweight with package manager
• ubuntu (~70MB) - Full environment

Node debugging:

kubectl debug node/<node-name> -it --image=ubuntu

Docker container debugging:

docker exec -it <container-id> sh

# If no shell available
docker run -it --pid=container:<container-id> \
           --net=container:<container-id> \
           busybox sh

For detailed container debugging patterns, see references/container-debugging.md.

Production Debugging

Production Debugging Principles

Golden rules:

1. Minimal performance impact - Profile overhead, limit scope
2. No blocking operations - Use non-breaking techniques
3. Security-aware - Avoid logging secrets, PII
4. Reversible - Can roll back quickly (feature flags, Git)
5. Observable - Structured logging, correlation IDs, tracing

Safe Production Techniques

1. Structured Logging

import logging
import json

logger = logging.getLogger(__name__)
logger.info(json.dumps({
    "event": "user_login_failed",
    "user_id": user_id,
    "error": str(e),
    "correlation_id": request_id
}))

2. Correlation IDs (Request Tracing)

func handleRequest(w http.ResponseWriter, r *http.Request) {
    correlationID := r.Header.Get("X-Correlation-ID")
    if correlationID == "" {
        correlationID = generateUUID()
    }
    ctx := context.WithValue(r.Context(), "correlationID", correlationID)
    log.Printf("[%s] Processing request", correlationID)
}

3. Distributed Tracing (OpenTelemetry)

from opentelemetry import trace

tracer = trace.get_tracer(__name__)

def process_order(order_id):
    with tracer.start_as_current_span("process_order") as span:
        span.set_attribute("order.id", order_id)
        span.add_event("Order validated")

4. Error Tracking Platforms

• Sentry - Exception tracking with context
• New Relic - APM with error tracking
• Datadog - Logs, metrics, traces
• Rollbar - Error monitoring

Production debugging workflow:

1. Detect - Error tracking alert, log spike, metric anomaly
2. Locate - Find correlation ID, search logs, view distributed trace
3. Reproduce - Try to reproduce in staging with production data (sanitized)
4. Fix - Create feature flag, deploy to canary first
5. Verify - Check error rates, review logs, monitor traces

For detailed production debugging patterns, see references/production-debugging.md.

Decision Framework