Testing Mwaa Workflow

Testing MWAA Workflows

AWS MCP server (optional but recommended): running the AWS CLI commands in this skill through the AWS MCP server gives sandboxed execution and audit logging. Every command here also works with the plain AWS CLI, so the skill does not require the MCP server or any MCP-only tools.

Test Amazon MWAA workflow execution end-to-end: trigger a run, poll it to a terminal state, and on failure drive a capped debug -> fix -> retest loop. This skill triggers and reads state only; it delegates every diagnosis to debugging-mwaa-workflow and every mutation (artifact edit, redeploy, environment create/update) to authoring-mwaa-workflow. Routes by flavor, then runs one shared spine.

Execution note — poll in discrete steps: whenever you wait for an AWS operation to reach a terminal or ready state, issue one status check per call and decide in your own loop whether to check again. Never block a single command or script on the wait (no while+sleep until done), regardless of the operation or how long it takes.

Guardrail — where this skill's own files live (MCP vs local install)

This skill can be loaded two ways, and they resolve the skill's own bundled files from different places. Determine how the skill was loaded before reading a reference:

• Loaded through the AWS MCP retrieve_skill tool: The skill is not installed on the local filesystem. You MUST fetch each reference via retrieve_skill with the file parameter (e.g. file="references/provisioned-testing.md") and read the returned content. Do NOT file_read these paths locally — they do not exist on disk.
• Installed locally (e.g. .kiro/skills/testing-mwaa-workflow/ or ~/.claude/skills/testing-mwaa-workflow/): Read the files from the local skill directory using relative paths.

This distinction applies only to the skill's own packaged files. User data and session artifacts are always read from and written to the user's working directory. Never fetch or write customer data through retrieve_skill.

Step 0: Detect Flavor, Mode, and Complexity

Flavor (reuse debugging's logic)

1. An environment name resolvable via aws mwaa get-environment -> Provisioned.
2. A workflow/... ARN or any aws mwaa-serverless context -> Serverless.
3. Neither signal -> ask: MWAA Provisioned (Python DAG) or MWAA Serverless (YAML workflow)?

Invocation mode

• Standalone — the user points at an existing, already-deployed target.
• Delegated — authoring-mwaa-workflow deployed and handed over the resolved target (env name + dag_id, or workflow ARN). Its "deploy and test now?" approval satisfies the first trigger confirmation only.

Complexity

• Simple — target already deployed and known-ready -> skip to Step 2.
• Standard — just deployed, readiness unknown -> full spine.
• Complex — retest inside an active fix loop.

Step 1: Readiness Check (before the first trigger)

Confirm the artifact is deployed and parse-ready. Do not trigger blind. This step is MANDATORY even when you plan to adopt an existing run — the readiness data (last_parsed_time, is_active, has_import_errors) feeds the freshness gate in Step 2.

• Provisioned: confirm dag_id present with has_import_errors: false, then apply the version-aware ready check via the /dags collection endpoint. Determine the poll window from the environment's configured scan interval: check get-environment -> AirflowConfigurationOptions for scheduler.dag_dir_list_interval (AF2) or dag_processor.refresh_interval (AF3). If unset, default to 300s. Poll up to that interval + 30s at 15s intervals (the per-DAG endpoint can lag). On AF2 (REST API v1) require is_active: true. On AF3 (REST API v2) the is_active field does not exist — require is_stale: false instead and never wait on is_active (it reads as absent/false forever). A DAG that is parsed but not yet ready will reject trigger attempts with an opaque RestApiClientException. Not ready in time -> hand to debugging-mwaa-workflow.
• Serverless: confirm WorkflowStatus is READY (via get-workflow --workflow-arn). Not ready -> hand to debugging-mwaa-workflow.

This readiness window is separate from the Step 3 run timeout. See references/provisioned-testing.md and references/serverless-testing.md for exact commands.

Step 2: Confirm, then Trigger (safety gate)

State the resolved target and classify its environment. Unless you are certain it is a development/test environment (name or tags clearly indicate dev/test), treat it as production: emit a prod warning and require explicit user confirmation before triggering. If the target is confirmed production (name or tags contain prod, prd, or production, or the user says so), require explicit approval at every state-changing step — each trigger, re-trigger, and clear/rerun — not just once. For a target you are certain is dev/test, require explicit user confirmation before the first trigger and each re-trigger.

This gate is already satisfied when the user has given explicit approval for the action: in delegated mode the authoring "deploy and test now?" approval covers the first trigger, and an explicit pre-authorization to trigger a specific run counts as that confirmation — do not re-ask when approval has already been given.

Before triggering, check for scheduler-created runs (see provisioned-testing reference). Apply the freshness gate before adopting any prior run:

• Delegated mode: always trigger fresh. Prior runs predate the deployment by definition — do not adopt regardless of state.
• Standalone mode: compare run.start_date against dag.last_parsed_time read from the /dags collection response the Step 1 readiness check already fetches (the collection returns last_parsed_time per DAG in both v1 and v2 — no per-DAG call needed). If start_date < last_parsed_time, the run tested a prior artifact version — treat as stale, trigger fresh. Only adopt runs where start_date >= last_parsed_time.

For fresh, non-stale runs that pass the gate: if a run already exists for the target interval, adopt it instead of POSTing. Monitor through Step 3.

On a retest, all run identifiers must be fresh (Provisioned: both dag_run_id and logical_date; Serverless: new start-workflow-run call). Before re-triggering, confirm every prior run is terminal — a still-running prior run can block the new one from starting.

See references for exact trigger commands and retest hygiene.

Step 3: Poll to Terminal

Poll the triggered run until it reaches a terminal state. See references for exact poll commands and error fallbacks.

Timeout handling:

• Provisioned with dagrun_timeout set: if elapsed time exceeds dagrun_timeout and the run is still not terminal, treat as failure -> Step 4. (dagrun_timeout is a DAG-level Airflow parameter and does not apply to MWAA Serverless.)
• Serverless: poll until the service returns a terminal state — no caller ceiling is needed, because the MWAA Serverless service enforces its own run cap and surfaces it as the TIMEOUT terminal state.
• Provisioned without dagrun_timeout: there is no DAG- or service-level deadline, so agree a maximum wait with the user before polling (they know the DAG's expected runtime; default to 1h if they have no preference). Poll until terminal OR the maximum wait elapses. On reaching the cap, stop polling and report — do not silently mark it failed, and do not keep polling unbounded:

  Run <run-id> has not reached a terminal state within the agreed <max-wait>. It may still be running — I have not failed it. Choose: (a) extend the wait, (b) inspect logs / the Airflow UI, or (c) treat this test as inconclusive.

Poll interval scales with elapsed time:

Pass = terminal SUCCESS only; no output-data inspection. On pass -> Step 6.