E2E Test

CRC E2E Test Runner

This skill runs CRC end-to-end tests with interactive feature and OS selection.

Arguments

User provided arguments: $ARGUMENTS

Available Features

The CRC e2e test suite includes these features:

• basic: Core CRC lifecycle tests (version, help, setup, start, stop, delete)
• config: Configuration management tests
• minimal: Minimal feature set tests (requires MicroShift bundle)
• story_openshift: OpenShift-specific story tests
• story_microshift: MicroShift-specific story tests
• story_application_deployment: Application deployment scenarios
• running_cluster_tests: Tests for running cluster operations
• cert_rotation: Certificate rotation tests (Linux only)
• story_manpages: Man page generation tests

Available OS Platforms

• linux: Linux platform tests
• darwin: macOS platform tests
• windows: Windows platform tests

Instructions

When this skill is invoked:

1. Parse arguments (if provided):

   • If arguments include a feature name and/or OS, use those
   • Otherwise, proceed to interactive selection

2. Interactive selection (if no arguments or partial arguments):

   Use AskUserQuestion tool with these specific questions:

   Question 1: Feature Selection

   • Header: "Feature"
   • Question: "Which e2e test feature(s) do you want to run?"
   • multiSelect: true (allow multiple features)
   • Options:
     • basic: "Core CRC lifecycle tests - setup, start, stop, delete, version, status (~30 min)"
     • config: "Configuration management and property validation tests (~15 min)"
     • minimal: "Quick minimal test suite for fast validation (~10 min, requires MicroShift bundle)"
     • story_openshift: "OpenShift-specific features and scenarios (~45 min)"
     • story_microshift: "MicroShift-specific features and scenarios (~45 min)"
     • story_application_deployment: "Application deployment workflows (~30 min)"
     • running_cluster_tests: "Tests for running cluster operations (~20 min, requires running cluster)"
     • cert_rotation: "Certificate rotation scenarios (~25 min, Linux only)"
     • story_manpages: "Man page generation and validation (~10 min)"

   Question 2: OS Platform Selection

   • Header: "Platform"
   • Question: "Which OS platform do you want to test?"
   • multiSelect: false (single selection)
   • Options:
     • current: "Use current platform ($(GOOS)) - Recommended"
     • linux: "Linux platform (most features supported)"
     • darwin: "macOS platform (arm64 and amd64)"
     • windows: "Windows platform (amd64 only)"

   Question 3: Bundle Location

   • Header: "Bundle"
   • Question: "Where is the CRC bundle located?"
   • multiSelect: false (single selection)
   • Options:
     • cache: "~/.crc/cache/crc_*.crcbundle (CRC cache directory) - Recommended"
     • downloads: "~/Downloads/crc_*.crcbundle"
     • custom: "Specify custom path"

   Question 4: Pull Secret Location

   • Header: "Pull Secret"
   • Question: "Where is the pull secret file located?"
   • multiSelect: false (single selection)
   • Options:
     • downloads: "~/Downloads/crc-pull-secret (default) - Recommended"
     • custom: "Specify custom path"

3. Handle custom paths (if user selected "custom" for any location):

   • For each "custom" selection, the user will provide the path via "Other" option
   • Extract the custom path from the user's response
   • Validate that custom paths exist before proceeding
   • Store paths for use in environment variables

4. Build CRC from source:

   • Run make cross to build CRC binaries for all platforms
   • This will create platform-specific binaries in the out/ directory:
     • Linux AMD64: out/linux-amd64/crc
     • Linux ARM64: out/linux-arm64/crc
     • macOS AMD64: out/macos-amd64/crc
     • macOS ARM64: out/macos-arm64/crc
     • Windows AMD64: out/windows-amd64/crc.exe
   • Display build progress to user
   • Verify the binary was built successfully for the target platform

5. Determine CRC binary directory:

   • Based on the selected platform, set the binary directory:
     • If platform is "current": detect using go env GOOS and go env GOARCH
     • If platform is "linux": use --crc-binary=out/linux-amd64/
     • If platform is "darwin": detect arch with uname -m (returns arm64 or x86_64); normalize x86_64 to amd64 and use --crc-binary=out/macos-<arch>/
     • If platform is "windows": use --crc-binary=out/windows-amd64/
   • Note: CRC_BINARY should be the directory path, not including the binary name itself
   • Verify the binary exists in the determined directory (e.g., out/linux-amd64/crc or out/windows-amd64/crc.exe)

6. Prepare test environment:

   • Based on bundle location selection:
     • If "cache": list available bundles in ~/.crc/cache and let user select or provide path
     • If "downloads": list available bundles in ~/Downloads and let user select or provide path
     • If "custom": ask user for the path
   • Verify the bundle file exists at the specified location
   • Verify the pull secret file exists at the specified location
   • Inform user about any missing prerequisites

7. Clean CRC state (IMPORTANT - Required for most tests):

   • If selected features include running_cluster_tests, skip crc cleanup and verify the cluster is running (crc status) instead.
   • Otherwise, run crc cleanup using the built binary (e.g., out/linux-amd64/crc cleanup)
   • This ensures tests start with a clean state (no existing VM, no stale configuration)
   • Critical for @basic tests which expect an unconfigured system
   • Note: Cleanup may require sudo password for removing system files:
     • /etc/udev/rules.d/99-crc-vsock.rules (udev rules)
     • /etc/modules-load.d/vhost_vsock.conf (vsock module config)
   • If cleanup fails due to missing sudo access at the end of test run, it's acceptable (cleanup failure in tests is cosmetic)
   • Use absolute paths (not tilde) for bundle and pull secret locations to avoid path expansion issues

8. Build test command:

   Construct the make command as follows:

   a. Start with base: make e2e

   b. Build tag filter:

   • If platform is "current": use $(GOOS) or detect with go env GOOS
   • If single feature: --godog.tags="<os> && @<feature>"
   • If multiple features: --godog.tags="<os> && (@feature1 || @feature2 || @feature3)"
   • Always include OS tag AND feature tag(s)

   c. Build environment variables based on user selections:

   • Always set CRC_BINARY to the platform-specific binary directory from step 5 (format: --crc-binary=/absolute/path/to/out/<platform>-<arch>/)
   • For bundle location:
     • IMPORTANT: Use absolute paths (not tilde ~) to avoid path expansion issues
     • If "cache": prompt user to select specific bundle from ~/.crc/cache or set BUNDLE_LOCATION=--bundle-location=/absolute/path/to/.crc/cache/<bundle_file>
     • If "downloads": prompt user to select specific bundle from ~/Downloads or set BUNDLE_LOCATION=--bundle-location=/absolute/path/to/Downloads/<bundle_file>
     • If "custom": set BUNDLE_LOCATION=--bundle-location=<absolute_custom_path>
   • For pull secret location:
     • IMPORTANT: Use absolute paths (not tilde ~)
     • If "downloads": set PULL_SECRET_FILE=--pull-secret-file=/absolute/path/to/Downloads/crc-pull-secret
     • If "custom": set PULL_SECRET_FILE=--pull-secret-file=<absolute_custom_path>
   • Note: All environment variables must include their flag prefix and use absolute paths

   d. Combine into final command:

   CRC_BINARY=--crc-binary=<binary_dir> [OTHER_ENV_VARS] make e2e GODOG_OPTS="--godog.tags=\"<constructed_tags>\""

   Note: <binary_dir> is the absolute directory path (e.g., /home/user/crc/out/linux-amd64/), not the full binary path

   e. Example commands:

   # Single feature on Linux with built binary and bundle from cache (using absolute paths)
   CRC_BINARY=--crc-binary=/home/user/crc/out/linux-amd64/ BUNDLE_LOCATION=--bundle-location=/home/user/.crc/cache/crc_libvirt_4.21.8_amd64.crcbundle PULL_SECRET_FILE=--pull-secret-file=/home/user/pull-secret make e2e GODOG_OPTS="--godog.tags=\"linux && @basic\""
   
   # Multiple features on macOS with built binary
   CRC_BINARY=--crc-binary=/Users/user/crc/out/macos-arm64/ BUNDLE_LOCATION=--bundle-location=/Users/user/.crc/cache/crc_vfkit_4.21.8_arm64.crcbundle make e2e GODOG_OPTS="--godog.tags=\"darwin && (@basic || @config || @minimal)\""

9. Execute tests:

   • Run the test command
   • Display progress and results to user
   • Provide clear feedback on pass/fail status
   • Note: If the final test cleanup step fails due to sudo password requirement, this is acceptable
     • The test suite runs its own cleanup at the end which may fail in non-interactive environments
     • Focus on the actual test results (scenarios/steps passed), not cleanup failure
     • A test is considered successful if all scenarios passed, even if final cleanup failed