Fetch Test Runs

Test Runs
============================================================

Test ID: openshift-tests:71c053c318c11cfc47717b9cf711c326
Job Contains: ['gcp', 'techpreview']
Include Successes: False
Runs Fetched: 18

Successes: 0, Failures: 18

Mass Failure Runs (>10 test failures in job): 4 of 18
  ⚠ These runs had many other test failures — this test may be caught up in a larger issue.

Sample Runs:

1. [FAIL] Job URL: https://prow.ci.openshift.org/view/gs/test-platform-results-public/logs/...
   Failed Tests in Job: 3
   Output: fail [k8s.io/kubernetes/test/e2e/apimachinery/discovery.go:145]: Fail to access...

2. [FAIL] [MASS FAILURE] Job URL: https://prow.ci.openshift.org/view/gs/test-platform-results-public/logs/...
   Failed Tests in Job: 47
   Output: fail [k8s.io/kubernetes/test/e2e/apimachinery/discovery.go:145]: Fail to access...

3. [PASS] Job URL: https://prow.ci.openshift.org/view/gs/test-platform-results-public/logs/...

... and 15 more runs

python3 fetch_test_runs.py "openshift-tests:abc"

{
  "success": false,
  "error": "Failed to connect to test runs API: Connection refused.",
  "test_id": "openshift-tests:abc",
  "requested_job_runs": 0,
  "include_success": false,
  "job_name_filters": null
}

Test Runs - FETCH FAILED
============================================================

Error: Failed to connect to test runs API: Connection refused.

The test runs API may not be available.

{
  "success": true,
  "test_id": "openshift-tests:abc",
  "requested_job_runs": 0,
  "include_success": false,
  "job_name_filters": null,
  "runs": []
}

python3 fetch_test_runs.py

Usage: fetch_test_runs.py <test_id> [job_run_ids] [options]

Arguments:
  test_id       Test identifier (e.g., 'openshift-tests:abc123')
  job_run_ids   Optional comma-separated list of Prow job run IDs

Options:
  --include-success    Include successful test runs (default: failures only)
  --job-contains       Filter by job name substring (repeatable for AND logic)
  --exclude-output     Strip output text from runs to reduce response size
  --output <path>      Write output to file instead of stdout
  --format json|summary   Output format (default: json)

script_path="plugins/ci/skills/fetch-test-runs/fetch_test_runs.py"
python3 "$script_path" "openshift-tests:bb3a7d828630760296ef203c5cacf708" --format json

script_path="plugins/ci/skills/fetch-test-runs/fetch_test_runs.py"
python3 "$script_path" "openshift-tests:bb3a7d828630760296ef203c5cacf708" --include-success --format json

# Assume regression_data is already fetched
test_id=$(echo "$regression_data" | jq -r '.test_id')
job_run_ids=$(echo "$regression_data" | jq -r '.sample_failed_jobs | to_entries[] | .value.failed_runs[] | .job_run_id' | tr '\n' ',' | sed 's/,$//')

# Fetch outputs for specific job runs
script_path="plugins/ci/skills/fetch-test-runs/fetch_test_runs.py"
output_data=$(python3 "$script_path" "$test_id" "$job_run_ids" --format json)

# Check success
if [ "$(echo "$output_data" | jq -r '.success')" = "true" ]; then
  echo "Successfully fetched runs"
fi

python3 plugins/ci/skills/fetch-test-runs/fetch_test_runs.py \
  "openshift-tests:71c053c318c11cfc47717b9cf711c326" \
  --format summary

# Fetch runs
output_data=$(python3 "$script_path" "$test_id" --format json)

# Extract all failure output messages
if [ "$(echo "$output_data" | jq -r '.success')" = "true" ]; then
  # Get all output texts from failed runs
  echo "$output_data" | jq -r '.runs[] | select(.success == false) | .output'

  # AI command will analyze these for:
  # - Similarity/consistency
  # - Common error patterns
  # - File references and API paths
  # - Root cause determination
fi

script_path="plugins/ci/skills/fetch-test-runs/fetch_test_runs.py"

# Get only GCP techpreview runs (both substrings must match, server-side)
python3 "$script_path" "openshift-tests:abc123" --include-success \
  --job-contains gcp --job-contains techpreview \
  --start-days-ago 28 --format json

# Get only metal upgrade runs
python3 "$script_path" "openshift-tests:abc123" \
  --job-contains metal --job-contains upgrade \
  --format summary

# Full job name also works (it's a substring of itself)
python3 "$script_path" "openshift-tests:abc123" \
  --job-contains "periodic-ci-openshift-release-master-nightly-4.22-e2e-gcp-ovn-techpreview" \
  --format json

script_path="plugins/ci/skills/fetch-test-runs/fetch_test_runs.py"

# Get the job with the most failures
most_failed_job="periodic-ci-openshift-release-master-nightly-4.22-e2e-metal-ipi-ovn"

# Fetch all runs (including successes) for this specific job, going back 28 days
job_history=$(python3 "$script_path" "$test_id" \
  --include-success \
  --job-contains "$most_failed_job" \
  --start-days-ago 28 \
  --format json)

# Analyze the run history
if [ "$(echo "$job_history" | jq -r '.success')" = "true" ]; then
  # Runs are returned newest to oldest
  # Iterate to find where failures started
  echo "$job_history" | jq -r '.runs[] | "\(.success) \(.url)"'

  # Look for transition from passing to failing
  # Find the first failure that's part of the current regression
fi