Quark Onnx Autosearch Pro

quark-onnx-autosearch-pro

Purpose

Drive Quark ONNX AutoSearchPro (Optuna-driven) on a .onnx model to discover the best quantization configuration — activation / weight specs, calibration method, CLE, AdaRound / AdaQuant hyperparameters — without hand-tuning. The recipe orchestrates intake, preset (or custom) search-space selection, reader construction, generation of a standalone autosearch script in the user's workspace, confirmed execution, and reporting of best_params.json plus Optuna study artifacts. Unlike single-shot PTQ this produces many candidate models, so preset and trial budget are explicit and the search is bounded.

Inputs

• Input .onnx model (with optional sibling .onnx_data).
• Calibration data: folder of representative samples or a Python CalibrationDataReader class. Optional separate evaluation reader.
• User goal: preset choice (ADVANCED_SEARCH / XINT8_SEARCH / A8W8_SEARCH / A16W8_SEARCH / custom dict), deployment target, search budget (n_trials, n_jobs, two_stage_search), and metric (built-in or custom callable).
• session_context.json (constraints.backend = "onnx"), env_context.json, workspace_context.json, plus onnx_install_result.json and quark_install_result.json. Optuna must be importable (pip install optuna).

Outputs: run_manifest.yaml

Records the generated autosearch script path, the exact python3 invocation, the preset name (or custom search-space hash), trial budget, Optuna study DB path, and the final best_params.json location.

Side artifacts written by AutoSearchPro into output_dir: auto_search.log, best_params.json, auto_search.db (resumable Optuna study), quantized_model_<trial>.onnx per trial, and opt_history.html / param_importance.html when plot_results=True.

Built in Step 4. Schema: run_manifest.schema.json

Interaction Flow

1. Intake — quark-onnx-model-intake → model_analysis.json
2. Preset / search space — pick preset or build custom → quant_plan.json (with search_space block + n_trials)
3. Reader + evaluator — confirm CalibrationDataReader and evaluator mode
4. Manifest — generate standalone autosearch script in user workspace + run_manifest.yaml; stop for user approval
5. Execute — run the confirmed script, report best_params.json

CRITICAL RULES

1. NEVER call AutoSearchPro(...).run() directly from this skill. Generate a standalone script the user reviews first. Searches can take hours and write many .onnx files — user must opt in.
2. NEVER skip preset confirmation. Presets default to optim_device = "cuda:0"; surface device gaps before execution.
3. NEVER silently fall back to CPU when CUDA/ROCm was requested. Hand off to quark-onnx-install / quark-onnx-debug.
4. NEVER write into the Quark repo. Generated scripts go to the user's workspace, not under examples/onnx/auto_search/ or tutorials/onnx/auto_search/. The shipped examples/onnx/auto_search/auto_search_pro_model.py is a template to read, not patch.
5. Pre-check optuna. AutoSearchPro imports it lazily; verify it imports in the user's env before writing the script.
6. Bound the budget. Refuse n_trials > 200 without explicit user opt-in — one .onnx per trial.
7. Validate custom search spaces with quark.onnx.quantization.auto_search.utils.validate_search_space. If n_trials > discrete_space_size without continuous fields, AutoSearchPro auto-clamps — surface it instead of letting it happen quietly.

See presets-reference.md for the preset table, deployment recommendations, custom-search-space rules, device gating snippet, evaluator modes, and sampler list.

Required Artifact Flow

Step 1 (Intake)        ──► model_analysis.json
Step 2 (Preset/space)  ──► quant_plan.json  (search_space + n_trials)
Step 3 (Reader)        ──► (informs script body)
Step 4 (Manifest)      ──► run_manifest.yaml + user_workspace/<name>_autosearch.py
Step 5 (Execute)       ──► best_params.json + auto_search.db + quantized_model_*.onnx

Step 1: Model Intake

Hand off to quark-onnx-model-intake. Stop if QDQ nodes already exist — AutoSearchPro requires an unquantized float .onnx. Add one recipe-specific line to the summary: "Search will produce ~N quantized .onnx files in output_dir — ensure ≥ N × <model_size> free."

>>> CHECKPOINT 1: Confirm model analysis

Step 2: Preset / Custom Search-Space Selection

Pick a built-in preset or accept a user-supplied search_space dict. See presets-reference.md for the full preset table and deployment-target recommendations.

For custom search spaces, run validate_search_space(...) in your head against the spec rules in the reference. Compute discrete_space_size and warn if n_trials would be clamped.

Decision table to show the user

>>> CHECKPOINT 2: User confirms preset, n_trials, devices, metric

Step 3: Calibration + Evaluation Reader

Reader patterns mirror quark-onnx-ptq-workflow Step 3 (vision: ImageDataReader; LLM: tokenizer-driven). AutoSearchPro wraps the reader with CachedDataReader — one pass over data is enough. For evaluator mode (built-in metric vs. custom search_evaluator(onnx_path) -> float) and direction semantics, see presets-reference.md. Pass a distinct eval_data_reader to score on a different slice; omit it to reuse calibration (AutoSearchPro warns).

>>> CHECKPOINT 3: Confirm reader + evaluator mode

Step 4: Manifest Generation

Translate the confirmed plan into a runnable script in the user's workspace and a run_manifest.yaml. Do not write into the Quark repo.