--- name: detection description: "Use for PaddleViT object detection workflows with DETR, Swin, or PVTv2: validate COCO data, select configs, build/train/evaluate models, reason about transforms, losses, post-processing, and run safe utility smokes. Excludes segmentation and generic export; cross-link deployment-and-operations for shared runtime concerns." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0 --- # PaddleViT object detection Use this route for standalone PaddleViT object detection under `object_detection/`: DETR, Swin, and PVTv2 with COCO boxes. It covers family/config selection, data preflight, transforms and targets, safe model-contract checks, checkpoint triage, training/evaluation boundaries, and backend-aware diagnosis. It does not own semantic segmentation, generic export/inference, quantization, or weight porting; route those requests to [deployment-and-operations](../deployment-and-operations/SKILL.md). The bundled helpers are self-contained and do not import the original source checkout, download data or weights, or modify a user-owned dataset. A source checkout is optional and is needed only for a source-model build or native candidate test. Details that are useful during a focused investigation live in [model overview](references/model-overview.md), [data formats](references/data-formats.md), [workflows](references/workflows.md), and [troubleshooting](references/troubleshooting.md). ## Route the request 1. Identify the family before changing a config or launching a command: - **DETR** for query-based, end-to-end detection with transformer decoder queries, Hungarian matching, and direct box post-processing. - **Swin** or **PVTv2** for hierarchical backbones feeding FPN, RPN, and RoI-style heads with anchors, proposals, and NMS. 2. If the request says only “PaddleViT detection,” ask which family and whether the goal is a utility check, source build, evaluation, or training. Do not mix family directories in one Python process: common modules such as `config`, `coco`, `box_ops`, and `utils` can resolve incorrectly. 3. Keep segmentation separate, even if a request mentions masks or a DETR import. For export, predictor, AMP, distributed, or shared runtime issues, use [deployment-and-operations](../deployment-and-operations/SKILL.md). ## Preflight before a source run Validate the **COCO root**, not a split directory. The expected root contains `annotations/instances_{train,val}2017.json` and the matching `train2017/` or `val2017/` image directory. Run the read-only helper from the skill root (replace placeholders with caller-owned values): ```bash python /scripts/check_coco_layout.py --split val ``` Add `--check-images` to decode referenced images and compare dimensions, or `--check-api` to parse the annotation through `pycocotools`. Use `--json` for a machine-readable report and `--demo` for a temporary one-image validator fixture. A failed check is a stop condition: repair the dataset outside this skill and rerun; do not download or silently substitute another dataset. Before importing a source model, run the bounded synthetic contract smoke: ```bash python /scripts/detection_model_smoke.py --model all --device cpu ``` This checks tiny DETR and four-level anchor-family shapes and finite values. It is not a source-model build, checkpoint test, COCO test, mAP result, or benchmark reproduction. Use `--device gpu:0` only when a GPU claim is required and the backend has already passed its environment probe. ## Select a source root and config The three projects are standalone. For a source-backed run, enter exactly one family directory and expose only that directory (and its intended parent) to imports: ```bash cd /object_detection/DETR # or Swin or PVTv2 export PYTHONPATH="$PWD:$PWD/..:${PYTHONPATH:-}" ``` Resolve configuration in this order: Python defaults, recursive YAML `BASE` files, then CLI overrides. Print and inspect the effective config before model construction. Check backbone output channels against FPN inputs, pyramid strides and anchor levels, class/category conventions, image divisibility, and DETR embedding-dimension/head divisibility. Swin/PVTv2 configs preserve the historical `ROI.NUM_ClASSES` spelling; do not replace it with a guessed key. The family-specific config and output/target details are in [model overview](references/model-overview.md) and [data formats](references/data-formats.md). ## CLI flag contract The family `main_single_gpu.py` and `main_multi_gpu.py` scripts use these short, single-dash options: ```text -cfg PATH YAML config (BASE files are recursively merged) -dataset coco dataset selector -data_path PATH COCO root, not train2017/val2017 -batch_size N per-process/per-GPU batch size -eval evaluation-only mode -pretrained PREFIX source appends .pdparams -resume PREFIX source expects .pdparams and .pdopt -last_epoch N resume epoch metadata -ngpus N configured multi-GPU worker count ``` `-cfg=...` and `-cfg ...` are both acceptable. Pass checkpoint **prefixes** without `.pdparams` when following the source loader. Inspect launcher shell files rather than executing them blindly: paths, visible devices, output prefixes, and training duration are caller-owned. A representative command shape is: ```bash cd /object_detection/DETR CUDA_VISIBLE_DEVICES=0 python main_single_gpu.py \ -cfg=./configs/detr_resnet50.yaml -dataset=coco \ -batch_size=1 -data_path= -eval \ -pretrained= ``` Treat real training/evaluation as expensive and data/checkpoint/GPU dependent; use the [workflow reference](references/workflows.md) for gated sequencing. ## Family contracts to preserve - **DETR:** expect logits shaped `[B,Q,C+1]`, normalized center-size boxes `[B,Q,4]`, optional auxiliary decoder outputs, and losses including classification, L1, and GIoU. Post-processing needs target sizes in `[height,width]` order and emits absolute `xyxy` boxes, scores, and labels. - **Swin/PVTv2:** expect hierarchical features, FPN/RPN/RoI losses during training, and post-NMS rows shaped `[label, score, xmin, ymin, xmax, ymax]` during evaluation. Their target path uses absolute boxes and contiguous classes; the two families share the broad head/neck contract but not every backbone channel/config value. - **All families:** COCO boxes begin as `[x,y,width,height]`; transformations must update geometry and area. A valid synthetic training fixture needs at least one non-empty target. Preserve original image IDs for COCO results. Full transform and output schemas are in the linked references. ## Verification tiers Choose the cheapest tier that answers the question and record command, device, Paddle version, config, and result: 1. `check_coco_layout.py` for root layout, JSON arrays, IDs, boxes, files, and optional Pillow/COCO-API checks. 2. `detection_model_smoke.py` for standalone tiny shape and finite-value contracts; a non-32-divisible size intentionally reports the padding caveat. 3. Native DETR box/transform/model tests when a source checkout and dependencies are available. They are evidence candidates, not runtime prerequisites; some are skipped or require an untracked fixture. 4. A one-batch source build/forward using one family root and a tiny local fixture, only after config/import checks pass. 5. Real COCO evaluation or training only with explicit data, checkpoint, GPU, and time approval. Never call a smoke or random-weight forward mAP evidence. CPU can validate parsing, layout, box utilities, and tiny diagnostics. It does not establish CUDA, AMP, distributed, or full detector claims. For those boundaries use [deployment-and-operations](../deployment-and-operations/SKILL.md). ## Stop and recover - **Missing files/API:** `-data_path` must be the root; verify exact split filenames. If `pycocotools` is absent, omit `--check-api` but do not claim COCO evaluation is verified. - **Import/YAML errors:** start a fresh process, isolate one family root, resolve `BASE` relative to its YAML, compare keys to that family's config, and inspect the final config. Do not mix module paths or claim a build that never constructed a model. - **Shape/checkpoint errors:** verify family, backbone/FPN channels, attention divisibility, query/class counts, head spelling, and checkpoint provenance. A prefix normally maps to `.pdparams`; resume also needs `.pdopt`. Do not reshape incompatible state dictionaries. - **Bad boxes/empty batches:** inspect crowd flags, category mapping, positive area, coordinate format, resize/flip/crop order, and image IDs. Keep one valid target in a fixture; an all-empty batch is not a valid smoke. - **NaN/Inf or no detections:** run CPU geometry checks, verify normalized DETR boxes or absolute Swin/PVTv2 boxes, then inspect thresholds, NMS, and loaded keys. Random weights cannot support an accuracy claim. - **CUDA OOM/unavailable:** lower per-GPU batch size or input scale for OOM. If CUDA is unavailable, report CPU-only partial evidence; never substitute it for a required GPU result. For multi-GPU, check visible devices, `-ngpus`, per-process batch semantics, NCCL, and rank gathering before retrying. See [troubleshooting](references/troubleshooting.md) for expanded recovery branches. Do not run network, full-native, long-training, or multi-GPU workflows as part of this route unless explicitly authorized.