Add New Model

Common pitfalls (from past PR reviews)

These recur often enough in review comments on past add-model PRs that they're worth checking proactively, not just waiting for a reviewer to catch them:

• Don't validate the same hparam/config assumption in both the Python conversion script and the C++ load path - pick one layer to own the check, duplicating it just adds maintenance surface.
• Optional hparams that are genuinely absent from some configs (e.g. a shared-expert count) should be read with an explicit optional/fallback accessor, not assumed present.
• Hparams that are actually load-bearing (the model produces wrong output or crashes without them, e.g. sliding_window_pattern, norm-eps) must hard-error if missing, not silently fall back to a default.
• Don't bake a default chat template into the C++ binary - inject it into the GGUF at conversion time instead, since one llm_arch can be reused by multiple fine-tunes with different templates, and a baked-in C++ default fails silently for those.
• Before writing a dedicated tool-call/output parser, check whether the existing autoparser already handles the template (test-chat-auto-parser <jinja> shows what it detects).
• Marking a custom EOS/closing-tag token as eot at conversion time isn't always sufficient - in long/agentic generations a model can emit the closing sequence as literal text instead of the token, so generation never stops on EOG and raw text leaks past the parser. Verify this case, not just the token path.
• If reusing or aliasing an existing pre-tokenizer for convenience, justify and test that choice explicitly - silent reuse is an easy source of subtle tokenizer bugs.
• Watch for excessive graph splits caused by building per-layer view/index tensors inside the layer loop - hoist tensors that don't vary per layer out of the loop (relevant if you hit GGML_SCHED_MAX_SPLIT_INPUTS).
• A custom KQ mask fed into flash attention must match FA's expected dtype - cast it to F16 before passing it to build_attn_mha when FA is enabled.
• When padding a custom KV-cache size to an alignment (e.g. GGML_PAD(..., 256)), apply the padding after all other size adjustments, not before - otherwise later logic can un-align it again.
• For non-standard cache/SWA (sliding-window-attention) semantics, override the dedicated hook (e.g. llama_model_n_swa()) rather than mutating hparams to fake the behavior - hparams may be read elsewhere for unrelated purposes.
• Don't ship unfinished or unverified speculative-decoding (e.g. MTP) scaffolding in the base model PR - if it hasn't actually been confirmed to work, pull it out and land it as its own follow-up.
• Conversion code should call into the base class's existing hparam logic (e.g. super().set_gguf_parameters()) rather than re-deriving it - large blocks of code that duplicate what TextModel/MmprojModel already provide will get flagged as redundant.
• Do constant tensor modifications (e.g. norm(1 + weight)) and permutations/chunking at conversion time, not in the graph - see HOWTO-add-model.md's "Prefer conversion-time tensor modifications" tip (Gemma 3 folds its 1 + into the weights, Qwen3-Next permutes in modify_tensors). Doing these at runtime in the graph is very likely to be rejected as over-complicated; if you genuinely can't do it at conversion time, open a discussion first explaining why rather than implementing it in the graph.
  • Exception: a plain weight * scale with a constant scale is usually better applied at inference time instead of being folded into the weight at conversion. The scale conceptually applies to the activation, not the weight, so folding it in can hurt numerical stability, and it shifts the weight's value range in a way that can make quantization worse.

Validation checklist

Reference: examples/model-conversion/README.md.

1. Convert to GGUF, then inspect/run both the original and converted tensors.
2. Run logits verification (original vs converted). If this model is a new version of an already-supported family, verify the previous version still passes logits verification first - numerical differences may be pre-existing, not caused by the new work. The tools to perform full logits validation are available in examples/model-conversion.
3. Quantize (including QAT variants if relevant) and re-verify.
4. Run perplexity evaluation (simple and full).
5. Sanity-check across tools/cli, tools/completion, tools/imatrix, tools/quantize, and tools/server.
6. CPU backend first; other backends (CUDA, Metal, ...) can be separate follow-up PRs per CONTRIBUTING.md (relaxed in this fork - fork-internal model work may bundle backends, but CPU-first still catches the most bugs cheapest).
7. Fork-specific: run the model with turbo KV cache types, not just f16/q8_0 - -ctk q8_0 -ctv turbo3 (and turbo2/turbo4), with flash attention. This exercises the rotation/padding path: head dims not a multiple of 128 must zero-pad correctly, and MLA/DeepSeek4 archs must use identical K/V types. Also quantize a copy to TQ4_1S and confirm it loads, decodes coherently, and runs on the CUDA kernels (TQ weights are arch-agnostic, but a new arch's graph must route them through the fused-TQ path, not the mmvq abort).
8. Re-review every changed file against the coding/naming guidelines in AGENTS.md (and CONTRIBUTING.md's "Coding guidelines"/"Naming guidelines" sections) - this is a separate pass from functional testing and is just as important: no forced line-wrapping, no unicode punctuation, minimal/non-redundant comments, snake_case naming (kebab-case for file names), matching indentation/brace style, etc.

Fork-specific additions (TurboQuant)

Take precedence over the upstream-facing rules where they conflict:

• Shared files carry turbo wiring: a new arch registers tensors in src/llama-arch.h/.cpp and builds its graph in src/models/<name>.cpp, but src/llama-graph.cpp also carries the fork's inverse-WHT post-processing (FA and non-FA paths) and src/llama-context.cpp carries the turbo FA auto-enable, head-dim padding, and MLA K/V equality checks. Do not refactor or reformat those blocks while adding a model; a structural change there that breaks turbo semantics is worse than a cosmetic diff.
• Turbo cache types are the point of this fork: a new arch must be verified with turbo KV (-ctv turbo3), not just default f16. If the model's head dims are not multiples of 128, exercise the zero-padding path explicitly. If it is MLA-family, K and V cache types must match and V rotation/padding is skipped - the existing DeepSeek4 handling is the reference.
• TQ weight types: new archs should be quantizable with llama-quantize ... TQ4_1S and the result must run (CUDA fused-TQ path; MoE models auto-disable CUDA graphs for TQ MUL_MAT_ID - do not try to re-enable).
• Rebase hygiene: keep src/models/<name>.cpp and conversion code structurally close to upstream style so the next upstream rebase stays clean; add a fork: tag comment on any deliberately fork-divergent block (see the code-review skill's TurboQuant section).
• Backend bundling: upstream's "CPU first, backends as follow-ups" is relaxed here for fork-internal work, but do not silently ship an unvalidated backend path - each claimed backend must have been run, not just compiled.
• Model files are the cleanest part of the tree: the fork's model support (src/models/, conversion/, gguf-py/) mostly matches upstream; if a merge/rebase produced stacked duplicates in gguf-py/gguf/constants.py (it has before - import gguf crashes), that is a separate cleanup, not part of the model PR.

Before opening a PR

• Run the code-review skill on the diff first - it catches the convention and scope issues reviewers flag most often, and it's recommended to do this locally before pushing the PR.
• Confirm the contributor can explain every changed line to a reviewer and is prepared to be asked about any of it - this is required regardless of how much of the code was AI-generated.
• Confirm they did a comprehensive manual review of the full diff, not just a skim.
• Fill in the AI-disclosure section of .github/pull_request_template.md describing how AI was used (do not omit or understate this).
• Do not write the PR description, commit message, GitHub issue/discussion text, or any reviewer replies yourself - the contributor writes these.