LLM inference and serving

Create and manage dstack presets: a toolkit that streamlines model inference optimization with agents, and a portable preset format.

dstackai/dstack

Always resolve Hugging Face models via model-shelf before any download.

alexziskind1/model-shelf

Maintain a persistent AI-tended research wiki. An agent skill from frankchu91/mindbase-llm-wiki.

frankchu91/mindbase-llm-wiki

Review and upgrade MetaX model support against a target vLLM revision and installed MACA components, recursively including model-dependent attention and kernels.

MetaX-MACA/vLLM-metax

CRITICAL - Guide for using Claudish CLI ONLY through sub-agents to run Claude Code with any AI model (OpenRouter, Gemini, OpenAI, local models).

MadAppGang/claudish

Build the standalone ZenDNN native library (zendnnl) from source with the alternate compute backends OFF (no oneDNN, libxsmm, parlooper, fbgemm), keeping AOCL DLP, which is the GEMM backend zendnnl…

amd/ZenDNN

Add a new hardware inference backend to AutoRound for deploying quantized models (e.g., CUDA/Marlin, Triton, CPU, HPU, ARK).

intel/auto-round

Check model compatibility with xinfer before loading. An agent skill from guoqingbao/xinfer.

guoqingbao/xinfer

Uses the Outlines library to constrain model output to a JSON schema, Pydantic model, regex or fixed set of choices when running local models.

Orchestra-Research/AI-Research-SKILLs

Use only for current stock/share prices, ticker quotes, and financial market movers (gainers, losers, most-traded shares).

zhongkaifu/TensorSharp

用于在 cc-router 仓库新增一个 LLM provider（即在 src-tauri/providers/ 下添加 YAML 描述符并完成配套的同步改动）。当用户说「加 provider」「接入 XX 厂商」「新增订阅源」「provider YAML」「让 cc-router 支持 OpenRouter/Together/Groq/Ollama 之类」时必须触发本…

finch-xu/cc-router

Regenerate the README UI screenshots for nyx-local-ai from the browser harness — build the webview, serve the harness scenes, capture and crop the images into docs/.

sthamann/nyx-local-ai

Builds an operator-level compute template for an LLM and estimates FLOPs and MFU for a serving shape, with tensor shapes and parallelism what-if checks.

BBuf/AI-Infra-Auto-Driven-SKILLS

Deploy BankMCP™ to a small server so it works in claude.ai and on the phone: Railway or Fly.io, volume, domain, setup page, connector.

noskillish/bankmcp

Add or update an in-repository vLLM-Omni model recipe with verified task, input, output, hardware, command, feature, and validation contracts.

vllm-project/vllm-omni

Review and refactor inference-runtime code using concrete rules for responsibility boundaries, state ownership, interfaces, asynchronous lifetimes, KV management, and maintainability.

ThinkFlowLab/vllm-rlt

The user wants to connect an LLM or vision provider, already has an API key, asks "can I use OpenAI/Anthropic/Gemini/OpenRouter", wants local Ollama, or needs different cheap and strong models.

oxbshw/watch-skill

Train a custom VideoHighlighter action or object model from a few videos the user provides — cut into samples, sort with CLIP, review contact sheets, build, train, install only if better.

Aseiel/VideoHighlighter

This skill should be used when working with DSPy.rb, a Ruby framework for building type-safe, composable LLM applications.

davekilleen/Dex

Deploys LLMs with vLLM for high-throughput serving, covering the OpenAI-compatible server, offline batch inference, monitoring and a Docker rollout.

Orchestra-Research/AI-Research-SKILLs

Cross-drive storage audit and cleanup. An agent skill from SethGammon/Citadel.

SethGammon/Citadel

Write or review Triton kernels for vLLM, with practical guidance for generated-code inspection, launch grids, indexing, specialization, tuning, and representative performance validation.

guqiong96/Lvllm

AIPC, AI Porting Conversion. An agent skill from qualcomm/qai-appbuilder.

qualcomm/qai-appbuilder

Reference desk for Nemotron 3 Nano / Llama-Nemotron Nano 3 — architecture, training data, recipes, evaluation, quantization, deployment.

NVIDIA-NeMo/Nemotron

Review and adapt vllmmetax/registry registrations, quantization configurations, CustomOps and kernel dispatch against a target vLLM revision and installed MetaX APIs.

MetaX-MACA/vLLM-metax

Interact with a Meshtastic LoRa mesh network through MESH-API — list nodes, read messages, send texts, and check connection status.

mr-tbot/mesh-api

Advisory guidance for Amazon EKS architecture and configuration decisions — compute strategy, networking, security, reliability, cost, autoscaling, observability, multi-tenancy, and upgrade planning.

aws-samples/appmod-blueprints

Add a new quantization data type to AutoRound (e.g., INT, FP8, MXFP, NVFP, GGUF variants).

intel/auto-round

Query Finelog logs and telemetry for Iris tasks, workers, profiles, training, vLLM, and cross-cluster forwarding.

marin-community/marin

Test LLM models served by xinfer for correctness, output quality, and performance.

guoqingbao/xinfer

Run a structured debate between agent CLIs (claude, codex, agy, grok) and the user's configured API or local models (OpenAI, Anthropic, Google, xAI, OpenRouter, Ollama and more) through the Quorum…

Detrol/quorum-cli

A skill your agent uses for web searches and current information lookups, finding sources, fact-checking, researching questions, comparing sources, or summarising web pages.

zhongkaifu/TensorSharp

Naming conventions for SGLang speculative decoding identifiers.

sgl-project/sglang

Validate changed quantization math, packed formats, and inference kernels against a Torch oracle; adjudicate near-tie mismatches before rejecting optimized arithmetic.

ModelCloud/GPTQModel

A skill your agent uses for setting up vector similarity search with pgvector for AI/ML embeddings, RAG applications, or semantic search.

timescale/pg-aiguide

Invoke ML models, run vector search, and connect to MCP servers from Databricks Apps.

databricks-solutions/databricks-apps-cookbook

Ship a new nyx-local-ai release end to end — bump versions consistently, run the quality gates (typecheck, smoke tests, package), install locally, tag and push so CI publishes the installer artifacts.

sthamann/nyx-local-ai

Optimizes and serves LLMs on NVIDIA GPUs with TensorRT-LLM, covering quantization, in-flight batching, multi-GPU parallelism and the trtllm-serve command.

Orchestra-Research/AI-Research-SKILLs

Set up BankMCP™ on this machine step by step: check Node, register the local MCP server, walk through the Enable Banking application, finish the setup page and connect the first bank.

noskillish/bankmcp

Deploys open models or custom weights from Model Garden to Agent Platform endpoints, checks deployment status and cleans up endpoints, confirming before any change.

google/skills

Guided workflow for adding a new model architecture to llama.cpp.

JakeATX/llamAmpere

Hardware-aware adaptive Hermes runtime using portable Turbofiles, total usable memory, owned native llama.cpp residency, stable auto/active:main/active:aux routes, and evidence-backed promotion.

SouthpawIN/turbofit

Run vLLM performance benchmark using synthetic random data to measure throughput, TTFT (Time to First Token), TPOT (Time per Output Token), and other key performance metrics.

vllm-project/vllm-skills

A skill your agent uses when the user wants to consult an AI persona "clone" for advice, strategy, analysis, or domain expertise—especially in startup, VC, tech, growth, HR, or business contexts.

team-attention/openclone

Serves AI models on AMD Instinct GPU hardware using vLLM. An agent skill from amd/skills.

amd/skills

Deploy LLM models on OCI using AI Quick Actions (AQUA) - single model, multi-model, stacked (LoRA), with GPU shape selection, vLLM configuration, streaming, and tool calling.

oracle/accelerated-data-science

Build vLLM (CPU) end-to-end with the oneDNN + ZenDNN (zen64) backend via direct integration (NO zentorch).

amd/ZenDNN

Review PRs and local changes for hsliuustc0106/vllm-rlt: Ouro engine and KV correctness, serving behavior, and BF16 accuracy/speed evidence.

ThinkFlowLab/vllm-rlt