New: 97,000+ skills indexed from GitHub repos with 100+ stars · every skill safety-checkedBrowse top skillsTop skills
Esc to closeBrowse all skills
Submit a skillBrowse skills
Submit a skillBrowse skills

Agent skill

Gptqmodel Tokenizer Normalization

by ModelCloud in ModelCloud/GPTQModel

Diagnose and correct GPT-QModel tokenizer initialization, tokenization normalization, special-token handling, prompt rendering, and chat-template problems.

Custom licenceAuto-check passedAI & LLM Engineering

Install Gptqmodel Tokenizer Normalization

skills CLI
$ npx skills add ModelCloud/GPTQModel --skill gptqmodel-tokenizer-normalization -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install ModelCloud/GPTQModel gptqmodel-tokenizer-normalization --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/ModelCloud/GPTQModel.git skills-src && mkdir -p .claude/skills && cp -r skills-src/.agents/skills/gptqmodel-tokenizer-normalization .claude/skills/gptqmodel-tokenizer-normalization && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
gptqmodel-tokenizer-normalization
GitHub stars
1.3k
Token cost
~1.1k tokens
SKILL.md length
510 words
Files
2
Skills in repo
2
Repo updated
First seen
Licence
Custom licence

At a glance

Diagnose and correct GPT-QModel tokenizer initialization, tokenization normalization, special-token handling, prompt rendering, and chat-template problems.

  • Works in 5 steps: Reproduce with the non-quantized model… → Capture the exact raw text, rendered… → Compare direct AutoTokenizer behavior,… → …
  • Evaluation quality suggests wrong token IDs
  • SKILL.md covers Diagnose the boundary, Implement upstream first and Consume the correction in…
  • Reaches github.com

What it does

Gptqmodel Tokenizer Normalization is an agent skill from ModelCloud/GPTQModel. Diagnose and correct GPT-QModel tokenizer initialization, tokenization normalization, special-token handling, prompt rendering, and chat-template problems. Use when inference or evaluation quality suggests wrong token IDs or prompts, when a model needs tokenizer load kwargs or compatibility patches, or when tokenizer behavior is being changed in GPT-QModel; reusable corrective behavior must be implemented, tested, versioned, pushed, and submitted upstream to github.com/ModelCloud/Tokenicer.

Its SKILL.md is about 1.1k tokens, which your agent loads only when the skill is triggered. The skill folder holds 2 other files (for example `agents/openai.yaml`).

It sits in AI & LLM Engineering, covering Database schema design, LLM inference and serving and Natural language processing. It works with GitHub, SGLang and vLLM. The repository describes itself as: LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.

When your agent uses it

  • Evaluation quality suggests wrong token IDs
  • A model needs tokenizer load kwargs
  • Compatibility patches
  • Tokenizer behavior is being changed in GPT-QModel

Example prompts

  • “/gptqmodel-tokenizer-normalization”

Workflow steps

5 steps, taken from the first numbered list in SKILL.md.

  1. Reproduce with the non-quantized model before attributing low scores or malformed output to quantization.
  2. Capture the exact raw text, rendered chat-template text, input IDs, attention mask, special-token settings, tokenizer class, tokenizer…
  3. Compare direct AutoTokenizer behavior, current Tokenicer.load(), and the proposed normalization on the same prompts. Include strings that…
  4. Run a small dense-model evaluation with the same task, generation settings, and evaluator used for the quantized model. If dense and…
  5. Separate reusable tokenizer behavior from evaluator-specific prompt construction. AutoTokenizer kwargs, tokenizer compatibility…

What it can do on your machine

Read from SKILL.md and the folder at commit d0e59f8. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    No scripts in the folder and no shell commands in SKILL.md.

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    Hosts in commands or code, which the agent is likely to contact:

    • github.com

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Gptqmodel Tokenizer Normalization loads about 1.1k tokens when it runs. Until then it costs about 132 tokens; SKILL.md has 510 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~132
When it runs · the whole SKILL.md, loaded when a task matches
~1.1k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

Its licence (Custom licence) doesn't allow us to republish the file, so here is its outline and opening line. It has 510 words (~1,104 tokens).

“Treat ModelCloud/Tokenicer as the source of truth for reusable tokenizer loading and normalization. GPT-QModel depends on Tokenicer; model loaders should consume its behavior rather than accumulate model-specific tokenizer patches.”

— opening of SKILL.md by ModelCloud, Custom licence
name
gptqmodel-tokenizer-normalization

Read the full SKILL.md on GitHub

Files

SKILL.md and 1 other file in .agents/skills/gptqmodel-tokenizer-normalization of ModelCloud/GPTQModel.

  • SKILL.md
  • agents/openai.yaml

Open the folder on GitHubat commit d0e59f8

Compare with similar skills

Gptqmodel Tokenizer Normalization next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Gptqmodel Tokenizer Normalization compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Gptqmodel Tokenizer Normalization this skillModelCloud/GPTQModel1.3k—~1.1kAutomated safety check: PassCustom licence
SageMaker Serving Image Selectionhuggingface/skills11k1 repos~4.6kAutomated safety check: PassApache-2.0
SGLang Structured ServingOrchestra-Research/AI-Research-SKILLs13k3 repos~2.9kAutomated safety check: PassMIT
Ascend Release Manager for vLLMvllm-project/vllm-ascend2.9k—~7.2kAutomated safety check: PassApache-2.0
Dstack Prototypingdstackai/dstack2.3k—~1.6kAutomated safety check: PassMPL-2.0
Debug InferenceNVIDIA/OpenShell15k—~1.9kAutomated safety check: PassApache-2.0

Similar skills

  • SageMaker Serving Image Selection

    huggingface/skills

    Official

    Chooses the right serving container and current image URI for deploying a Hugging Face model to a SageMaker endpoint, preferring Hugging Face images over generic ones.

    11k GitHub starsUsed in 1 repo~4.6k tokens
    AI & LLM EngineeringAuto-check passed

  • SGLang Structured Serving

    Orchestra-Research/AI-Research-SKILLs

    Covers serving LLMs with SGLang, whose RadixAttention reuses cached prefixes, and constraining output to JSON, regex or grammar for agent and tool-calling workloads.

    13k GitHub starsUsed in 3 repos~2.9k tokens
    AI & LLM EngineeringAuto-check passed

  • Ascend Release Manager for vLLM

    vllm-project/vllm-ascend

    Runs the end-to-end vLLM Ascend release process: opens the release checklist and feedback issues, scans for release-blocking bugs and test coverage gaps, and generates release notes and announcements.

    2.9k GitHub stars~7.2k tokensUpdated today
    AI & LLM EngineeringAuto-check passed

  • Dstack Prototyping

    dstackai/dstack

    Use with the dstack skill for model-serving work when the image, serving command, resources, backend/fleet choice, or service behavior is not proven.

    2.3k GitHub stars~1.6k tokensUpdated today
    AI & LLM EngineeringAuto-check passed

  • Debug Inference

    NVIDIA/OpenShell

    Official

    Debug inference clients that use an attached provider and its native endpoint, including hosted APIs and host-local Ollama, vLLM, SGLang, TRT-LLM, LM Studio, or NIM.

    15k GitHub stars~1.9k tokensUpdated today
    AI & LLM EngineeringAuto-check passed

  • Graphsignal

    graphsignal/graphsignal

    Profile AI inference workloads (vLLM, SGLang, TensorRT-LLM, PyTorch, any GPU application) with the Graphsignal profiler and read the results from its local /signals JSON endpoint.

    257 GitHub stars~6.2k tokensUpdated 9 days ago
    AI & LLM EngineeringAuto-check passed

More from ModelCloud/GPTQModel

  • Gptqmodel Torch Oracle Numerics

    ModelCloud/GPTQModel

    Validate changed quantization math, packed formats, and inference kernels against a Torch oracle; adjudicate near-tie mismatches before rejecting optimized arithmetic.

    1.3k GitHub stars~1.4k tokensUpdated today
    Auto-check passed

Works with

Categories

Questions about Gptqmodel Tokenizer Normalization

What does Gptqmodel Tokenizer Normalization do?

Diagnose and correct GPT-QModel tokenizer initialization, tokenization normalization, special-token handling, prompt rendering, and chat-template problems. Gptqmodel Tokenizer Normalization is an agent skill from ModelCloud/GPTQModel. Diagnose and correct GPT-QModel tokenizer initialization, tokenization normalization, special-token handling, prompt rendering, and chat-template problems.

When should I use Gptqmodel Tokenizer Normalization?

Gptqmodel Tokenizer Normalization fits situations like: evaluation quality suggests wrong token IDs; A model needs tokenizer load kwargs; compatibility patches; tokenizer behavior is being changed in GPT-QModel.

How do I install Gptqmodel Tokenizer Normalization in Claude Code?

Run `npx skills add ModelCloud/GPTQModel --skill gptqmodel-tokenizer-normalization -a claude-code`. Or copy the skill folder (.agents/skills/gptqmodel-tokenizer-normalization in ModelCloud/GPTQModel) into .claude/skills/gptqmodel-tokenizer-normalization in your project. Claude Code loads it when a task matches its description.

How do I install Gptqmodel Tokenizer Normalization in Codex?

Run `npx skills add ModelCloud/GPTQModel --skill gptqmodel-tokenizer-normalization -a codex`. Or copy the skill folder (.agents/skills/gptqmodel-tokenizer-normalization in ModelCloud/GPTQModel) into .agents/skills/gptqmodel-tokenizer-normalization in your project. Codex loads it when a task matches its description.

Can I use Gptqmodel Tokenizer Normalization in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add ModelCloud/GPTQModel --skill gptqmodel-tokenizer-normalization -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/gptqmodel-tokenizer-normalization, .gemini/skills/gptqmodel-tokenizer-normalization, .github/skills/gptqmodel-tokenizer-normalization and .opencode/skills/gptqmodel-tokenizer-normalization in your project.

What does Gptqmodel Tokenizer Normalization need to run?

SKILL.md names no scripts, command-line tools or credentials: Gptqmodel Tokenizer Normalization is instructions for the agent only.

Does Gptqmodel Tokenizer Normalization access the network?

SKILL.md names 1 domain. In commands or code: github.com; the agent is likely to contact it when it follows the instructions. This is read from the text; nothing was executed.

Is Gptqmodel Tokenizer Normalization safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Gptqmodel Tokenizer Normalization use?

Gptqmodel Tokenizer Normalization has a licence file (the repository's licence) that doesn't match a standard licence. Read it on GitHub before reusing the skill.

How many tokens does Gptqmodel Tokenizer Normalization use?

About 1.1k tokens (SKILL.md is roughly 4.4k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Gptqmodel Tokenizer Normalization?

Skills that share tags, products or a category with Gptqmodel Tokenizer Normalization: SageMaker Serving Image Selection (huggingface/skills, 11k stars), SGLang Structured Serving (Orchestra-Research/AI-Research-SKILLs, 13k stars), Ascend Release Manager for vLLM (vllm-project/vllm-ascend, 2.9k stars) and Dstack Prototyping (dstackai/dstack, 2.3k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Gptqmodel Tokenizer Normalization?

ModelCloud (a GitHub organization) maintains it in ModelCloud/GPTQModel, which has 1,270 GitHub stars. The repository holds 2 skills in this directory. The repository was last updated on October 7, 2026.

Source: ModelCloud/GPTQModel on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.