Run E2E Test

Run E2E Test

When To Use

Use this skill when the user wants to manually replay a cloud-provider-azure e2e test case against a live Kubernetes cluster. The skill parses the Go test source, translates each step to CLI commands (kubectl / az), and guides interactive execution.

Prerequisites

• az login completed (active Azure session)
• Valid KUBECONFIG pointing at the target cluster
• kubectl available on $PATH
• python3 available on $PATH

Inputs

Workflow

Replace <SKILL_DIR> with the path to this skill directory.

Step 1 — Confirm Cluster Context

Unless --skip-context-check is passed:

python3 <SKILL_DIR>/scripts/check_context.py

This displays the current kube context, cluster endpoint, and Azure subscription. Ask the user to confirm this is the correct cluster before proceeding. If the user says no, stop and ask them to switch context.

Step 2 — Analyze the Test

python3 <SKILL_DIR>/scripts/analyze_test.py <test-file-path> [--test-name "<test name>"] [--json]

This parses the Go test file and outputs a structured plan with:

• variables: Go const/var declarations (deployment names, service names, labels, ports) extracted from the file — use these to resolve ${VAR} placeholders in emitted commands
• setup: BeforeEach steps (namespace, deployment, Azure client init)
• test_cases: list of It(...) blocks, each with ordered steps
• teardown: AfterEach cleanup steps

Each step includes:

• type: k8s_create, k8s_update, k8s_delete, k8s_wait, az_create, az_read, az_delete, az_wait, connectivity_check, assertion, phase
• description: human-readable explanation
• command: suggested CLI command(s)
• wait: polling parameters if applicable
• verify: what to check in the output

Use --json to get machine-readable JSON output instead of the human-readable format:

python3 <SKILL_DIR>/scripts/analyze_test.py tests/e2e/network/ensureloadbalancer.go --json

If --test-name is provided, only that test case is included. Otherwise all test cases are listed.

Variable Resolution

The variables field in the JSON output lists Go constants and variable declarations extracted from the test file header. Use them to populate ${VAR} placeholders in the emitted commands. For example, if the output contains "testBaseName": "service-lb", substitute ${DEPLOYMENT_NAME} with the value derived from that constant. Some variables may reference other Go symbols — in that case, read the test source to resolve the chain.

Step 3 — Review the Plan

Present the extracted plan to the user. For each test case show:

1. Setup steps (namespace, deployment, pre-created Azure resources)
2. Test body steps in order
3. Verification checks
4. Cleanup steps

Ask the user to confirm before executing. They may choose to skip certain steps or modify parameters.

Step 4 — Execute Setup

Run the setup steps:

1. Create namespace: kubectl create namespace <generated-name>
2. Create deployment: generate YAML from the test's deployment manifest and kubectl apply -f -
3. Wait for pods: kubectl wait --for=condition=ready pod -l <labels> -n <ns> --timeout=5m
4. Pre-create Azure resources (if needed): PIPs, subnets, IP prefixes via az network ... commands

Track all created resources for cleanup.

If any setup step fails:

1. Run the diagnostic command from the Error Handling section
2. Report the error with diagnostic output
3. Ask the user whether to clean up now (jump to Step 6) or keep the environment for manual investigation
4. Do not proceed to Step 5

Step 5 — Execute Test Steps

For each step in the test body:

1. Create/update K8s resources: generate YAML, kubectl apply -f -
2. Wait for conditions: poll with the timeouts from the test
   • Service IP: kubectl get svc -n <ns> <name> -o jsonpath='{.status.loadBalancer.ingress[*].ip}'
   • Poll every 10s, timeout per test constants (typically 5m for Standard LB)
3. Verify Azure state: run az commands and check output matches assertions
4. Connectivity checks: kubectl exec <exec-pod> -- nc -vz -w 4 <ip> <port>
5. Report results: for each assertion, show PASS/FAIL with actual vs expected

Before executing, read references/INDEX.md (the patterns index). For each step in the plan, match the step's description keywords and step_type against the index to identify which sections of references/patterns.md are needed. Read only those sections using the line ranges from the index. Do not read the entire patterns.md file.

On step failure, follow the Failure Flow waterfall in the Error Handling section. Key behaviors:

• Transient errors (429, 409, 5xx, timeout): retry per the retry policy
• Poll timeout (condition never met): record as assertion FAIL
• Assertion/verification mismatch: record FAIL with actual vs expected, run diagnostics, then ask the user whether to (a) continue, (b) pause for investigation, (c) abort to cleanup, or (d) re-check
• Non-recoverable errors: stop and jump to Step 6

Step 6 — Cleanup

Run cleanup in reverse order:

1. Delete K8s services
2. Delete K8s deployments
3. Delete pre-created Azure resources (PIPs, subnets, prefixes)
4. Delete namespace

Always offer cleanup even if a step failed. Track which resources were actually created to avoid deleting things that don't exist.

If a cleanup step fails:

1. Retry the deletion once after 30s
2. If it still fails, log the resource as orphaned and continue to the next cleanup step
3. Include all orphaned resources in the Step 7 report

Step 7 — Report

Summarize the execution:

Always include:

• Total steps executed: N
• Passed assertions: N
• Failed assertions: N

Include if any failures occurred:

• Skipped steps: N (with reason for each)
• For each failure:
  • Step description
  • Expected vs actual
  • Diagnostic output (if collected)
• Infrastructure errors encountered
• Re-checked assertions: N (list step name, attempts before pass or final fail)

Always include:

• Resources created: list
• Resources cleaned up: list
• Orphaned resources: list (if any cleanup failed)

Bundled Resources

• scripts/check_context.py — display and validate current kube/Azure context
• scripts/analyze_test.py — parse Go e2e test file into structured replay plan
• references/INDEX.md — generated keyword and step-type index into patterns.md; read this first to find which sections to load
• references/patterns.md — comprehensive mapping of Go e2e test patterns to kubectl and az CLI commands, including manifest templates and polling patterns
• scripts/gen_patterns_index.py — regenerate references/INDEX.md from keyword annotations in patterns.md

Error Handling

Error Categories

Retry Policy

Retry transient errors before classifying as a failure: