New: 98,000+ skills indexed from GitHub repos with 100+ stars · every skill safety-checkedBrowse top skillsTop skills
Esc to closeBrowse all skills
Submit a skillBrowse skills
Submit a skillBrowse skills

Agent skill

Inference Evaluation Quantization

by VectorSpaceLab in VectorSpaceLab/AREX-Skill

Use torchtune generation, Eleuther evaluation, and quantization workflows safely after checkpoints exist.

BSD-3-ClauseAuto-check passedAI & LLM Engineering

Install Inference Evaluation Quantization

skills CLI
$ npx skills add VectorSpaceLab/AREX-Skill --skill inference-evaluation-quantization -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install VectorSpaceLab/AREX-Skill inference-evaluation-quantization --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/VectorSpaceLab/AREX-Skill.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/repositories/repo-skills/torchtune/sub-skills/inference-evaluation-quantization .claude/skills/inference-evaluation-quantization && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
inference-evaluation-quantization
GitHub stars
328
Token cost
~1k tokens
SKILL.md length
365 words
Files
5 (incl. scripts, references)
Skills in repo
157
Repo updated
First seen
Licence
BSD-3-Clause

At a glance

Use torchtune generation, Eleuther evaluation, and quantization workflows safely after checkpoints exist.

  • Tasks that involve LLM inference and serving
  • SKILL.md covers Safe Default, Route By Task, API And Recipe Facts and Boundaries And Cross-Links
  • Runs Python scripts from its folder; calls python

What it does

Inference Evaluation Quantization is an agent skill from VectorSpaceLab/AREX-Skill. Use torchtune generation, Eleuther evaluation, and quantization workflows safely after checkpoints exist.

Its SKILL.md is about 1k tokens, which your agent loads only when the skill is triggered. The skill folder holds 6 other files, including scripts and reference files (for example `references/checkpoint-flow.md`, `references/troubleshooting.md` and `references/workflows.md`).

It sits in AI & LLM Engineering, covering LLM inference and serving. The repository describes itself as: A Skill Library for Automated Machine Learning. The licence is BSD-3-Clause.

When your agent uses it

  • Tasks that involve LLM inference and serving

Example prompts

  • “/inference-evaluation-quantization”

Requirements

  • Python 3

What it can do on your machine

Read from SKILL.md and the folder at commit ac3fe1a. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Ships 1 file in scripts/ (Python), which the agent can run.

    Shell commands in SKILL.md call:

    • python

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    No URLs in SKILL.md.

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Inference Evaluation Quantization loads about 1k tokens when it runs, and up to ~6.8k if it reads all its reference files. Until then it costs about 35 tokens; SKILL.md has 365 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~35
When it runs · the whole SKILL.md, loaded when a task matches
~1k
With references · SKILL.md plus every file in references/, read only if the agent opens them
~6.8k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.

SKILL.md

The full file from VectorSpaceLab/AREX-Skill at commit ac3fe1a, republished under its BSD-3-Clause licence (© VectorSpaceLab). 365 words, ~1,021 tokens.

Download SKILL.mdSave it as .claude/skills/inference-evaluation-quantization/SKILL.md (or your agent's skills folder). This skill also uses 4 other files; get the full folder from GitHub.
name
inference-evaluation-quantization
description
Use torchtune generation, Eleuther evaluation, and quantization workflows safely after checkpoints exist.
disable-model-invocation
true
metadata.disco-role
operating
license
BSD 3-Clause

inference-evaluation-quantization

Use this sub-skill when an agent already has model checkpoints and needs to run or plan torchtune generation, EleutherAI Eval Harness evaluation, post-training quantization conversion, or evaluation/generation of quantized checkpoints.

Do not use it to choose or launch training recipes. Route training setup to ../post-training-recipes/SKILL.md, model component details to ../models-and-modules/SKILL.md, and checkpointing internals to ../training-utilities-and-rlhf/SKILL.md when available.

Safe Default

Generation, evaluation, and quantization can download gated model assets, allocate GPUs, compile kernels, write checkpoints, or require optional packages. Build and inspect commands first; only execute after the user confirms checkpoint files, tokenizer files, credentials, device, dtype, output directory, and optional dependencies.

Use the bundled command builder to construct a non-executing command:

bash
python sub-skills/inference-evaluation-quantization/scripts/build_inference_eval_command.py generate \
  ./custom_generation_config.yaml \
  --override checkpointer.checkpoint_dir=./runs/lora/epoch_0 \
  --override checkpointer.checkpoint_files=[model-00001-of-00002.safetensors,model-00002-of-00002.safetensors] \
  --override tokenizer.path=./runs/lora/epoch_0/original/tokenizer.model \
  --print-notes

The script prints tune run ...; it never executes recipes, imports recipes, reads checkpoints, downloads models, or touches GPUs.

Route By Task

  • For command/config construction for generate, eleuther_eval, quantize, and quantize-then-evaluate/generate sequences, read references/workflows.md.
  • For deciding between base checkpoints, merged LoRA weights, adapter-only outputs, and quantized checkpoint compatibility, read references/checkpoint-flow.md.
  • For optional dependency, checkpointer/quantizer, tokenizer, prompt, GPU/memory, dtype, and output-dir failures, read references/troubleshooting.md.
Show full SKILL.md (191 more words)Show less

API And Recipe Facts

  • Use tune run generate --config <config> for the stable generation recipe, tune run eleuther_eval --config <config> for EleutherAI harness evaluation, and tune run quantize --config <config> for torchtune quantization conversion.
  • Do not import recipes; recipes are launched through tune run, copied with tune cp, inspected with tune cat, or executed through the CLI/runpy path.
  • The public generation API includes torchtune.generation.generate(model, prompt, max_generated_tokens, pad_id=0, temperature=1.0, top_k=None, stop_tokens=None, rng=None, compiled_generate_next_token=None), sample(logits, temperature=1.0, top_k=None, q=None), and generate_next_token(...).
  • The generation recipe supports single-GPU generation; the Eleuther recipe supports single-GPU evaluation and quantization for text-only models.
  • Eleuther evaluation requires the optional lm-eval package in the supported harness range; quantization workflows require torchao-backed quantizer components.
  • Use ../cli-and-config/SKILL.md for registry discovery, tune cp, tune cat, tune validate, and OmegaConf override syntax.
  • Use ../post-training-recipes/SKILL.md before this sub-skill when the model checkpoint has not been produced yet.
  • Use ../training-utilities-and-rlhf/SKILL.md for deeper checkpointer behavior, precision utilities, device utilities, logging, and checkpoint state conventions.
  • Use ../models-and-modules/SKILL.md for selecting model builders or understanding model architecture internals.
  • Keep generated runtime instructions self-contained; do not rely on source-repo docs, recipe files, tests, or local checkout paths being available.

© VectorSpaceLab, BSD-3-Clause. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 4 other files (scripts, references) in skills/repositories/repo-skills/torchtune/sub-skills/inference-evaluation-quantization of VectorSpaceLab/AREX-Skill.

  • SKILL.md
  • references/checkpoint-flow.md
  • references/troubleshooting.md
  • references/workflows.md
  • scripts/build_inference_eval_command.py

Open the folder on GitHubat commit ac3fe1a

Compare with similar skills

Inference Evaluation Quantization next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Inference Evaluation Quantization compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Inference Evaluation Quantization this skillVectorSpaceLab/AREX-Skill328—~1kAutomated safety check: PassBSD-3-Clause
SageMaker Serving Image Selectionhuggingface/skills11k1 repos~4.6kAutomated safety check: PassApache-2.0
Fine-Tuning ExpertJeffallan/claude-skills12k1 repos~1.7kAutomated safety check: PassMIT
Hugging Face Local Model Evalshuggingface/skills11k2 repos~1.6kAutomated safety check: PassApache-2.0
Qwen Mtp GgufR6410418/Jackrong-llm-finetuning-guide1.7k—~1.7kAutomated safety check: PassMIT
CI Fails Buildkiteguqiong96/Lvllm4642 repos~349Automated safety check: PassApache-2.0

Similar skills

  • SageMaker Serving Image Selection

    huggingface/skills

    Official

    Chooses the right serving container and current image URI for deploying a Hugging Face model to a SageMaker endpoint, preferring Hugging Face images over generic ones.

    11k GitHub starsUsed in 1 repo~4.6k tokens
    AI & LLM EngineeringAuto-check passed

  • Fine-Tuning Expert

    Jeffallan/claude-skills

    Guides LLM fine-tuning with LoRA and QLoRA through Hugging Face PEFT, from dataset validation and training checks to adapter merging, quantization and deployment.

    12k GitHub starsUsed in 1 repo~1.7k tokens
    AI & LLM EngineeringAuto-check passed

  • Hugging Face Local Model Evals

    huggingface/skills

    Official

    Runs evaluations of Hugging Face Hub models on local hardware with inspect-ai or lighteval, and helps choose between vLLM, Transformers and accelerate backends.

    11k GitHub starsUsed in 2 repos~1.6k tokens
    AI & LLM EngineeringAuto-check passed

  • Qwen Mtp Gguf

    R6410418/Jackrong-llm-finetuning-guide

    Complete agent-ready workflow for Qwen-family MTP or nextn GGUF conversion and release.

    1.7k GitHub stars~1.7k tokensUpdated 2 mo ago
    AI & LLM EngineeringAuto-check passed

  • CI Fails Buildkite

    guqiong96/Lvllm

    Fetch and diagnose vLLM Buildkite CI failure logs. An agent skill from guqiong96/Lvllm.

    464 GitHub starsUsed in 2 repos~349 tokens
    AI & LLM EngineeringAuto-check passed

  • Subwave LLM Bench

    perminder-klair/subwave

    Benchmark and compare LLM models for SUB/WAVE's on-air calls — track picks, segments, listener requests, DJ scripts, banter, and programme beats — in both candidate-pool and agent modes, using…

    1.4k GitHub stars~2.4k tokensUpdated today
    AI & LLM EngineeringAuto-check: notes

More from VectorSpaceLab/AREX-Skill

All 157 skills in this repo

  • Agent Lightning

    VectorSpaceLab/AREX-Skill

    Use this repo skill for Agent Lightning package tasks: authoring trainable agents, tracing rewards and spans, running LightningStore/Trainer loops, using agl CLI services, choosing examples, and…

    328 GitHub stars~1.4k tokensUpdated 1 mo ago
    Auto-check passed

  • Agent Tools

    VectorSpaceLab/AREX-Skill

    A skill your agent uses when configuring LiteLLM for MCP tools, A2A agents, Claude Code/Cursor agent gateway traffic, MCP auth/OAuth, tool permissions, semantic filtering, or agent-specific proxy…

    328 GitHub stars~1.2k tokensUpdated 1 mo ago
    Auto-check passed

  • Agents And Awel

    VectorSpaceLab/AREX-Skill

    Build and debug DB-GPT agents, tools, skills, teams, and AWEL workflows, including deterministic local DAG runs and HTTP-trigger topology without assuming an LLM, credential, or external service.

    328 GitHub stars~2.3k tokensUpdated 1 mo ago
    Auto-check passed

  • Agents And Middleware

    VectorSpaceLab/AREX-Skill

    Work on the actively maintained LangChain v1 agent package: initchatmodel, createagent, structured output, tools, middleware, embeddings initialization, provider routing, and agent runtime…

    328 GitHub stars~1.2k tokensUpdated 1 mo ago
    Auto-check passed

  • Agents Workflows

    VectorSpaceLab/AREX-Skill

    A skill your agent uses for giskard.agents async chat workflows, tools, prompt templates, structured outputs, retries, rate limiting, embeddings, and optional LiteLLM backend.

    328 GitHub stars~500 tokensUpdated 1 mo ago
    Auto-check passed

  • Alphafold3

    VectorSpaceLab/AREX-Skill

    A skill your agent uses for AlphaFold 3 input preparation, prediction command planning, output interpretation, and Python API inspection.

    328 GitHub stars~1.2k tokensUpdated 1 mo ago
    Auto-check passed

Categories

Questions about Inference Evaluation Quantization

What does Inference Evaluation Quantization do?

Use torchtune generation, Eleuther evaluation, and quantization workflows safely after checkpoints exist. Inference Evaluation Quantization is an agent skill from VectorSpaceLab/AREX-Skill. Use torchtune generation, Eleuther evaluation, and quantization workflows safely after checkpoints exist.

When should I use Inference Evaluation Quantization?

Inference Evaluation Quantization fits situations like: tasks that involve LLM inference and serving.

How do I install Inference Evaluation Quantization in Claude Code?

Run `npx skills add VectorSpaceLab/AREX-Skill --skill inference-evaluation-quantization -a claude-code`. Or copy the skill folder (skills/repositories/repo-skills/torchtune/sub-skills/inference-evaluation-quantization in VectorSpaceLab/AREX-Skill) into .claude/skills/inference-evaluation-quantization in your project. Claude Code loads it when a task matches its description.

How do I install Inference Evaluation Quantization in Codex?

Run `npx skills add VectorSpaceLab/AREX-Skill --skill inference-evaluation-quantization -a codex`. Or copy the skill folder (skills/repositories/repo-skills/torchtune/sub-skills/inference-evaluation-quantization in VectorSpaceLab/AREX-Skill) into .agents/skills/inference-evaluation-quantization in your project. Codex loads it when a task matches its description.

Can I use Inference Evaluation Quantization in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add VectorSpaceLab/AREX-Skill --skill inference-evaluation-quantization -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/inference-evaluation-quantization, .gemini/skills/inference-evaluation-quantization, .github/skills/inference-evaluation-quantization and .opencode/skills/inference-evaluation-quantization in your project.

What does Inference Evaluation Quantization need to run?

Going by SKILL.md and its folder, Inference Evaluation Quantization needs Python for the scripts in its folder and the command-line tools its instructions call (python). Our summary lists: Python 3.

Does Inference Evaluation Quantization access the network?

SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.

Is Inference Evaluation Quantization safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. The check reads SKILL.md only: the scripts in the folder are not scanned, so read them before running anything.

What licence does Inference Evaluation Quantization use?

Inference Evaluation Quantization is published under the BSD-3-Clause licence (declared in SKILL.md). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Inference Evaluation Quantization use?

About 1k tokens (SKILL.md is roughly 4.1k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 5.8k tokens, read only when the agent opens those files.

What are the alternatives to Inference Evaluation Quantization?

Skills that share tags, products or a category with Inference Evaluation Quantization: SageMaker Serving Image Selection (huggingface/skills, 11k stars), Fine-Tuning Expert (Jeffallan/claude-skills, 12k stars), Hugging Face Local Model Evals (huggingface/skills, 11k stars) and Qwen Mtp Gguf (R6410418/Jackrong-llm-finetuning-guide, 1.7k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Inference Evaluation Quantization?

VectorSpaceLab (a GitHub organization) maintains it in VectorSpaceLab/AREX-Skill, which has 328 GitHub stars. The repository holds 157 skills in this directory. The repository was last updated on September 3, 2026.

Source: VectorSpaceLab/AREX-Skill on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.