Check Model

Check Model — Pre-Load Compatibility Audit for xinfer

Phase 0: Gather Model Information

Collect model config and tensor info. Accept any of:

If tensor info is missing, ask the user to provide it. They can get it by clicking any .safetensors file in the HuggingFace model page and copying the tensor tree.

For local models, extract tensor info with:

import json, struct, sys, glob, os
path = sys.argv[1]
for sf in sorted(glob.glob(os.path.join(path, "*.safetensors"))):
    with open(sf, "rb") as f:
        n = struct.unpack("<Q", f.read(8))[0]
        header = json.loads(f.read(n))
    for k, v in sorted(header.items()):
        if k != "__metadata__":
            print(f"{k}\t{v.get('shape')}\t{v.get('dtype')}")

Phase 1: Parse Config and Identify Model Type

Extract from config.json:

Core parameters

Hybrid (Qwen3.5/Qwen3Next) parameters

MoE parameters

Quantization config

Normalize quant_method

Apply the same normalization as QuantConfig::normalize_compressed_tensors():

Phase 2: Validate Tensor Format Against Quantization Config

For each layer type, check that tensor names and dtypes match the expected format.

2a. Determine which layers are quantized vs skipped

Parse the ignore list from quantization_config. Layers in the ignore list should have BF16/FP16 weights (weight tensor only). Layers NOT in the ignore list should have quantized tensors.

The ignore list supports:

• Literal paths: "model.language_model.layers.0.linear_attn.in_proj_qkv"
• Regex patterns: "re:.*linear_attn.*"
• Glob-style wildcards: "model.visual*", "mtp.layers.0*"

2b. Format-specific tensor checks

Unquantized (BF16/FP16)

Expected tensors per linear layer:

• weight — dtype BF16 or F16, shape [out_dim, in_dim]
• bias (optional) — dtype BF16 or F16

Check: No extra scale/packed tensors should be present.

FP8 (quant_method == "fp8")

Expected tensors per linear layer:

• weight — dtype U8 (F8_E4M3), shape [out_dim, in_dim]
• weight_scale or weight_scale_inv — dtype F32, shape [out_dim/by, in_dim/bx] where [by, bx] = weight_block_size (default [128, 128])
• bias (optional)

Check: weight_block_size must have exactly 2 elements. Scale dimensions must match ceil(out_dim/by) x ceil(in_dim/bx).

NVFP4 — ModelOpt format (quant_method == "modelopt" + quant_algo == "NVFP4")

Expected tensors per quantized linear layer:

• weight — dtype U8, shape [out_dim, in_dim/2] (packed FP4, 2 values per byte)
• weight_scale — dtype F8_E4M3 (U8), shape [out_dim, in_dim/16] (group_size=16)
• weight_scale_2 — dtype F32, scalar (global weight scale, direct multiplier)
• input_scale — dtype F32, scalar (activation scale)

Check: weight shape dim1 must be exactly in_dim/2. Scale dim1 must be in_dim/16.

NVFP4 — Compressed-tensors format (quant_method == "compressed-tensors" + nvfp4 format)

Expected tensors per quantized linear layer:

• weight_packed — dtype U8, shape [out_dim, in_dim/2]
• weight_scale — dtype F8_E4M3 (U8), shape [out_dim, in_dim/16]
• weight_global_scale — dtype F32, scalar or [1] (divisor, inverted at load time)
• input_global_scale — dtype F32, scalar or [1] (divisor, inverted at load time)

Check: Same shape rules as ModelOpt, but different tensor names.

MXFP4 (quant_method == "mxfp4" or compressed-tensors with mxfp4)

Expected tensors per quantized linear layer:

• weight_packed or blocks — dtype U8, shape [out_dim, in_dim/2]
• weight_scale or scales — dtype U8 (F8_E8M0), shape [out_dim, in_dim/32] (group_size=32)

Check: Scale dim1 must be in_dim/32.

GGUF

GGUF models are self-contained (no config.json). Weight tensor names use blk.{i} prefix mapped to model.layers.{i}. Quantization is per-tensor via GGML dtypes (Q4_K, Q6_K, Q8_0, etc.).

Check: Not applicable for safetensors checks. GGUF has its own loader path via QLinear / QMatMul.

2c. Loader path tensor name resolution

The xinfer loaders try tensor names in priority order. Verify the model's tensors match at least one:

Flag any mismatch where the model uses a tensor name not in the priority list.

2d. Hybrid model (GDN) quantization detection

For Qwen3.5/Qwen3Next models with quantization config, the GatedDeltaNet layer has its own quantization detection (is_weight_quantized) that checks each linear_attn sublayer independently:

If a linear_attn sublayer is in the ignore list and has only BF16 weight, the detection returns false, and the layer loads as unquantized. Verify this matches the tensor info.