Physical AI Image Attribute Augmentation

Procedure

1. Determine the input source.

   • For local data, validate before upload:

     python scripts/upload_images.py --path /path/to/crops --validate-only

   • Then upload while preserving the hierarchy:

     python scripts/upload_images.py \
       --path /path/to/crops --destination-path image-attribute-augmentation/my-run

   • For an existing storage URL, use it unchanged after confirming it contains person-ID subdirectories. Each immediate subdirectory of input_path is treated as one person ID, and its images are combined into a single horizontal strip per person.

2. Select service mode.

   • external requires explicit VLM, LLM, and image-edit endpoint URLs.
   • internal lets the DAG's service lifecycle deploy all three services in-cluster.
   • Choose service mode independently from controller placement. A local controller may use external inference endpoints.
   • Keep nested service mode and output directory consistent with the top level.
   • On Kubernetes each deployed endpoint claims one GPU from k8s_gpu_1, so internal mode needs at least three allocatable GPUs (more if any replicas value is raised); external mode needs none for inference.

3. Read payload-contract.md, then render a payload from the values collected above. Do not copy checked-in dev or CI payloads — they contain deployment- specific endpoint URLs and bucket paths that must not be inherited by user runs.

   External:

   python scripts/payload.py render \
     --input-path s3://bucket/input/person-crops/ \
     --output-directory s3://bucket/output/image-attribute-augmentation/ \
     --service-mode external \
     --vlm-url https://vlm.example/v1 \
     --llm-url https://llm.example/v1 \
     --image-edit-url https://image-edit.example/v1 \
     --max-imgs 10 --num-augmentation 3 \
     --variable-distribution assets/variable-distribution.json \
     --output /tmp/iaa-payload.json

   Internal:

   python scripts/payload.py render \
     --input-path s3://bucket/input/person-crops/ \
     --output-directory s3://bucket/output/image-attribute-augmentation/ \
     --service-mode internal \
     --max-imgs 10 --num-augmentation 3 \
     --output /tmp/iaa-payload.json

   Show the user the rendered payload (or its validated contents) and get explicit confirmation before proceeding. Only continue to preflight and triggering if they confirm; if they want changes, re-render and re-confirm.

4. Preflight the DAG through the Airflow API. Check that the DAG is loaded, required pools have slots, and controller pods are healthy — see airflow-direct-api.md#preflight-direct-path. Confirm presence only; never print credential values.

5. Submit exactly one DAG run. Pass the payload from step 3 as conf.payload — see airflow-direct-api.md#trigger-a-run for the full request shape. Record and return the dag_run_id, input path, output directory, and service mode.

6. Immediately after triggering — without waiting to be asked — monitor the run until it reaches a terminal state (success or failed). Poll the Airflow API every 60–120 seconds:

   # Poll run state
   RESPONSE=$(curl -s -H "Authorization: Bearer $TOKEN" \
     "$AIRFLOW_URL/api/v2/dags/$DAG_ID/dagRuns/$RUN_ID")
   RESPONSE="$RESPONSE" python3 -c "import json, os; print(json.loads(os.environ['RESPONSE'])['state'])"

   For a per-task breakdown when state is running or failed, see airflow-direct-api.md.

   Stop polling as soon as the run state is success or failed. Use the polling loop that fits your runtime — a shell while loop, a background process, or a tool-native scheduler. Do not block the user waiting for each poll; report state changes as they occur.

   Tell the user they can also watch progress live in the Airflow UI. make port-forward runs in the foreground and never exits, so start it as a background job — and prefer that the user runs it in their own terminal, since an agent-owned forward dies with the session. Resolve the host's real address rather than reporting a placeholder or localhost, which is meaningless from another machine:

   HOST_IP=$(hostname -I | awk '{print $1}')
   echo "Airflow UI: http://$HOST_IP:8080"

   Default credentials are admin/admin, defined in deploy/values.yaml under airflow.createUserJob.defaultUser (not webserver.defaultUser). Update them before production use.

   For a full per-task breakdown see airflow-direct-api.md.

   To stop an in-progress run: open the Airflow UI, find the active DagRun, locate the running task, and mark it Failed (task menu → Mark Failed). This triggers the DAG's shutdown path, cleaning up Deployments, Services, and GPU pods. Do not delete the DagRun or the DAG — that bypasses cleanup and leaves stale cluster resources.

7. After the run reaches success or failed, ask the user: "Would you like to download and analyze the results?" Do not download automatically — wait for confirmation.

   If the user confirms, use whatever AWS credentials are already available in the shell environment (standard AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY / AWS_DEFAULT_REGION, an AWS profile, or instance role). Never ask the user to paste credentials into the prompt. Run artifacts live under <output_directory>/<run_id>/, where <output_directory> is the payload value and <run_id> is the dag_run_id from step 5. The final dataset is in augmented_dataset/:

   aws s3 sync "<output_directory>/<run_id>/augmented_dataset/" /tmp/iaa-results/
   python scripts/summarize_results.py --results-dir /tmp/iaa-results/augmented_dataset

   To inspect intermediate augmented images instead, sync <output_directory>/<run_id>/cosmos/ and read output_metadata.json from each <person_id>/<augmentation_index>/ folder.

   Read outputs.md before interpreting files.

Guardrails

• Never use default endpoint URLs, bucket paths, or input paths from the codebase or checked-in payloads. Always ask the user for every deployment-specific value before building a payload. If a required value is missing, stop and ask — do not substitute a guess.
• Preserve explicit user inputs and endpoint/model selections throughout the session.
• Do not submit if payload validation, local dataset validation, or Airflow preflight fails.
• Do not show AWS credentials, Airflow bearer tokens, or S3 signed URLs.
• Do not start multiple runs unless the user explicitly requests them.
• Ask for a dataset location if none was supplied; this workflow has no implicit demo dataset.
• Do not invent augmentation-only or labeling-only DAG IDs — only the DAG listed above exists.
• Only offer a platform whose manifest exists and whose DAG is loaded in Airflow.

References

• Read setup-and-preflight.md for environment, storage, and policy requirements.
• Read payload-contract.md when creating or changing a payload.
• Read outputs.md when retrieving or interpreting results.
• Read troubleshooting.md after validation, API, or runtime errors.
• Read airflow-direct-api.md for all Airflow interactions: preflight, triggering, monitoring, per-task breakdown, and log retrieval.