AI & LLM Engineering · SGLang

Replay-first debug flow for SGLang serving problems. An agent skill from sgl-project/sglang.

sgl-project/sglang

Unified LLM torch-profiler triage skill for sglang, vllm, TensorRT-LLM, and TokenSpeed.

sgl-project/sglang

Chooses the right serving container and current image URI for deploying a Hugging Face model to a SageMaker endpoint, preferring Hugging Face images over generic ones.

huggingface/skills

Debug hanging issues in SGLang distributed inference (TP/PP/DP/EP).

sgl-project/sglang

Write, calibrate, and debug the prefill-vs-decode logprob (KL) consistency tests in sglang -- the two independent conditions a zero requires (every operator batch-invariant, and the two paths…

sgl-project/sglang

Plans and audits Day-0 SGLang support for a new model release: scope, architecture gaps, PR order, validation gates and sanitized public evidence.

BBuf/AI-Infra-Auto-Driven-SKILLS

Code style for SGLang large classes Scheduler, TokenizerManager, and ModelRunner: frozen-code conventions and init orchestration style.

sgl-project/sglang

Use with the dstack skill for model-serving work when the image, serving command, resources, backend/fleet choice, or service behavior is not proven.

dstackai/dstack

Debug inference clients that use an attached provider and its native endpoint, including hosted APIs and host-local Ollama, vLLM, SGLang, TRT-LLM, LM Studio, or NIM.

NVIDIA/OpenShell

Reads SGLang or vLLM startup logs to show where GPU memory went and estimates how many concurrent requests fit at common token lengths.

BBuf/AI-Infra-Auto-Driven-SKILLS

Profile AI inference workloads (vLLM, SGLang, TensorRT-LLM, PyTorch, any GPU application) with the Graphsignal profiler and read the results from its local /signals JSON endpoint.

graphsignal/graphsignal

Clean up noisy startup warnings and spurious prints in SGLang server logs.

guqiong96/Lsglang

驱动 One-Eval 对 API 或本地模型做端到端评测，覆盖纯文本、多模态、代码生成、函数调用和 Agent benchmark。当用户想评测模型在一个或多个 benchmark 上的表现、比较分数、补充 metric，或生成图文评测报告时使用本 skill。

OpenDCAI/One-Eval

Trigger the bot-cherry-pick workflow for a batch of merged PRs onto a release branch and monitor each run to completion.

sgl-project/sglang

Analyzes Torch Profiler traces from SGLang, vLLM and TensorRT-LLM servers into kernel attribution, overlap and fusion tables.

BBuf/AI-Infra-Auto-Driven-SKILLS

Diagnose and correct GPT-QModel tokenizer initialization, tokenization normalization, special-token handling, prompt rendering, and chat-template problems.

ModelCloud/GPTQModel

Add a new model export format to AutoRound (e.g., autoround, autogptq, autoawq, gguf, llmcompressor).

intel/auto-round

A skill your agent uses when benchmarking denoise latency or profiling a diffusion bottleneck in SGLang.

sgl-project/sglang

Looks up public original architecture diagrams for named LLM, vision-language, MoE, diffusion and OCR models and returns the image with its source attribution.

BBuf/AI-Infra-Auto-Driven-SKILLS

A skill your agent uses when quantizing a diffusion DiT with NVIDIA ModelOpt and making the resulting FP8 or NVFP4 checkpoint loadable, verifiable, and benchmarkable in SGLang Diffusion.

sgl-project/sglang

Step-by-step tutorial for adding a new lightweight JIT CUDA kernel to sglang's jitkernel module

guqiong96/Lsglang

Naming conventions for SGLang speculative decoding identifiers.

sgl-project/sglang

Validate changed quantization math, packed formats, and inference kernels against a Torch oracle; adjudicate near-tie mismatches before rejecting optimized arithmetic.

ModelCloud/GPTQModel

Guides reinforcement-learning post-training of LLMs with slime, which pairs Megatron-LM training with SGLang rollouts, including GRPO runs on GLM, Qwen3 and Llama 3 models.

Orchestra-Research/AI-Research-SKILLs

Sets up a structured debugging session for a Dynamo bug — pull the report from a Linear ticket, GitHub issue, or pasted text, capture the environment, create a persistent worklog markdown file, and…

ai-dynamo/dynamo

A skill your agent uses when choosing the fastest SGLang Diffusion flags for a model, GPU, and VRAM budget.

sgl-project/sglang

Step-by-step tutorial for adding a heavyweight AOT CUDA/C++ kernel to sgl-kernel (including tests & benchmarks)

sgl-project/sglang

Call this skill when you need to debug CUDA crashes in SGLang using kernel API logging

sgl-project/sglang

Requirements for the SGLang scripted runtime, chiefly when to add (vs not add) a harness API.

sgl-project/sglang

Investigate consistently failing SGLang CI tests by extracting the failure signature from scheduled or rerun workflows, bisecting the passing/failing commit window, checking runner or hardware…

sgl-project/sglang

A skill your agent uses when adding a new diffusion model or Diffusers pipeline to SGLang.

sgl-project/sglang

Run RecIF beam×concurrency eval and FlashRec vs SGLang/vLLM/TRT-LLM baselines.

sohu-mptc/FlashRec

Bump Heron version via the VERSION-file SSOT. An agent skill from Netis/heron.

Netis/heron

Covers serving LLMs with SGLang, whose RadixAttention reuses cached prefixes, and constraining output to JSON, regex or grammar for agent and tool-calling workloads.

Orchestra-Research/AI-Research-SKILLs

Trains LLMs with reinforcement learning using verl, from ByteDance's Seed team, with GRPO, PPO and other algorithms and swappable training and rollout backends.

Orchestra-Research/AI-Research-SKILLs

How SGLang's runtime configuration and process-global state are organized (RuntimeContext tiers, publish + namespace config bags, the pristine ServerArgs seed, override entry points…

sgl-project/sglang

Run and troubleshoot the inference backends on the DGX Spark pair that FastLLM proxies to — starting or stopping models with vLLM, SGLang or sparkrun, choosing memory and speculative-decoding…

azrtydxb/Fastllm-proxy

Sync documentation with project state using ICAV workflow. An agent skill from Netis/heron.

Netis/heron

Breaks LLM torch profiler traces down by forward pass, layer and kernel, with timing tables and Perfetto time ranges for the layers you want to inspect.

BBuf/AI-Infra-Auto-Driven-SKILLS

Workflow for upgrading/integrating cache-dit in SGLang diffusion (multimodalgen): DBCache, DMD calibrator, TaylorSeer, SVDQuant DQ; porting upstream PRs and resolving conflicts against the…

sgl-project/sglang

Add or verify a model's chat prompt rendering and tokenization in rust/sglang-processor so it matches SGLang's Python serving path exactly (same prompt text, same token ids).

sgl-project/sglang

Autonomously optimizes end-to-end LLM inference throughput on AMD Instinct GPUs and reports a validated gain, using the Hyperloom multi-agent optimizer.

amd/skills

Compares SGLang, vLLM, TensorRT-LLM and TokenSpeed on one model and workload, searching server flags to find the best deployment command within a latency SLA.

BBuf/AI-Infra-Auto-Driven-SKILLS

Run a 4-hour multi-node Hyperloom Qwen3-30B-A3B optimization (Infera PD-disaggregated or RayJob aggregated) with --nodes 2 and sglang MoE tuning on MI325X.

AMD-AGI/Hyperloom

Benchmarks LLM inference and drives GPU kernel optimization with Magpie.

amd/skills

A skill your agent uses when an SGLang, vLLM, TensorRT-LLM, or TokenSpeed serving/model optimization task needs prior model-family PR evidence.

BBuf/AI-Infra-Auto-Driven-SKILLS

Audit the existing test tree and CI configuration for improvements, using a catalog of patterns previously applied in this repo.

sgl-project/sglang

Upgrade focused runtime dependencies in AReaL. An agent skill from areal-project/AReaL.

areal-project/AReaL