Ascend Model Adapter for vLLM

vLLM Ascend Model Adapter

Overview

Adapt Hugging Face or local models to run on vllm-ascend with minimal changes, deterministic validation, and single-commit delivery. This skill is for both already-supported models and new architectures not yet registered in vLLM.

Read order

1. Start with references/workflow-checklist.md.
2. Read references/multimodal-ep-aclgraph-lessons.md (feature-first checklist).
3. If startup/inference fails, read references/troubleshooting.md.
4. If checkpoint is fp8-on-NPU, read references/fp8-on-npu-lessons.md.
5. Before handoff, read references/deliverables.md.

Hard constraints

• Never upgrade transformers.
• Primary implementation roots are fixed by Dockerfile:
  • /vllm-workspace/vllm
  • /vllm-workspace/vllm-ascend
• Start vllm serve from /workspace with direct command by default.
• Default API port is 8000 unless user explicitly asks otherwise.
• Feature-first default: try best to validate ACLGraph / EP / flashcomm1 / MTP / multimodal out-of-box.
• --enable-expert-parallel and flashcomm1 checks are MoE-only; for non-MoE models mark as not-applicable with evidence.
• If any feature cannot be enabled, keep evidence and explain reason in final report.
• Do not rely on PYTHONPATH=<modified-src>:$PYTHONPATH unless debugging fallback is strictly needed.
• Keep code changes minimal and focused on the target model.
• Final deliverable commit must be one single signed commit in the current working repo (git commit -sm ...).
• Keep final docs in Chinese and compact.
• Dummy-first is encouraged for speed, but dummy is NOT fully equivalent to real weights.
• Never sign off adaptation using dummy-only evidence; real-weight gate is mandatory.

Execution playbook

1) Collect context

• Confirm model path (default /models/<model-name>; if environment differs, confirm with user explicitly).
• Confirm implementation roots (/vllm-workspace/vllm, /vllm-workspace/vllm-ascend).
• Confirm delivery root (the current git repo where the final commit is expected).
• Confirm runtime import path points to /vllm-workspace/* install.
• Use default expected feature set: ACLGraph + EP + flashcomm1 + MTP + multimodal (if model has VL capability).
• User requirements extend this baseline, not replace it.

2) Analyze model first

• Inspect config.json, processor files, modeling files, tokenizer files.
• Identify architecture class, attention variant, quantization type, and multimodal requirements.
• Check state-dict key prefixes (and safetensors index) to infer mapping needs.
• Decide whether support already exists in vllm/model_executor/models/registry.py.

3) Choose adaptation strategy (new-model capable)

• Reuse existing vLLM architecture if compatible.
• If architecture is missing or incompatible, implement native support:
  • add model adapter under vllm/model_executor/models/;
  • add processor under vllm/transformers_utils/processors/ when needed;
  • register architecture in vllm/model_executor/models/registry.py;
  • implement explicit weight loading/remap rules (including fp8 scale pairing, KV/QK norm sharding, rope variants).
• If remote code needs newer transformers symbols, do not upgrade dependency.
• If unavoidable, copy required modeling files from sibling transformers source and keep scope explicit.
• If failure is backend-specific (kernel/op/platform), patch minimal required code in /vllm-workspace/vllm-ascend.

4) Implement minimal code changes (in implementation roots)

• Touch only files required for this model adaptation.
• Keep weight mapping explicit and auditable.
• Avoid unrelated refactors.

5) Two-stage validation on Ascend (direct run)

Stage A: dummy fast gate (recommended first)

• Run from /workspace with --load-format dummy.
• Goal: fast validate architecture path / operator path / API path.
• Do not treat Application startup complete as pass by itself; request smoke is mandatory.
• Require at least:
  • startup readiness (/v1/models 200),
  • one text request 200,
  • if VL model, one text+image request 200,
  • ACLGraph evidence where expected.

Stage B: real-weight mandatory gate (must pass before sign-off)

• Remove --load-format dummy and validate with real checkpoint.
• Goal: validate real-only risks:
  • weight key mapping,
  • fp8/fp4 dequantization path,
  • KV/QK norm sharding with real tensor shapes,
  • load-time/runtime stability.
• Require HTTP 200 and non-empty output before declaring success.
• Do not pass Stage B on startup-only evidence.