Best LLM inference and serving skills, page 6

Delegate coding to OpenCode CLI (multi-LLM open-source coding agent)

taracodlabs/aiden

Extend, test, debug, or integrate the Model Download microservice codebase.

open-edge-platform/edge-ai-libraries

Download and convert AI models using the Model Download microservice.

open-edge-platform/edge-ai-libraries

Explains how model files, checkpoints, GGUF quantization, and the Model Manager work in HOT-Step CPP.

scragnog/HOT-Step-CPP

End-to-end ONNX PTQ workflow for AMD Quark — from a .onnx file (and calibration data) to a quantized .onnx output.

amd/Quark

Automatically fetch InferenceX benchmark data and generate daily performance reports for LLM inference on various hardware (NVIDIA, AMD, etc.).

ascend-ai-coding/awesome-ascend-skills

[omh] Self-hosted LLM serving on GPUs: choose the serving engine and quantization from decision tables, prepare deployment as an idempotent runbook with observed-only verification, and measure the…

rlaope/oh-my-hermes

Stand up vLLM or SGLang serving on Jetson, using upstream vLLM on Thor and Orin JetPack 7.2+, and NVIDIA-AI-IOT vLLM on older Orin.

NVIDIA/skills

Add EAGLE-3 or draft-model speculative decoding to a Jetson vLLM server when TPOT is the bottleneck.

NVIDIA/skills

Launch NON-AGENTIC (standard) data generation — plain vLLM/API completion generation with NO Harbor agent loop or Daytona sandboxes.

open-thoughts/OpenThoughts-Agent

Launch agentic Harbor evals through the OT-Agent unified eval listener (eval/unifiedevallistener.py) on any cluster: select models (queryunevaledmodels.py / priority lists), wire the pinggy…

open-thoughts/OpenThoughts-Agent

Deeper backend map of overbae — module layout, celery queue topology, the span-only tracing model and its API surface, capabilities and toml sync, behaviour-keyed scoring, auth and guests, model…

overmind-core/overmind

Train ML models on Databricks. An agent skill from databricks/databricks-agent-skills.

databricks/databricks-agent-skills

Interactively guide setup for connecting the Claude Code CLI to a local Ollama LLM on Mac.

receptron/mulmoclaude

Canonical tempo-sync/rate/quantize divisions from src/audio/MusicTime.h (4 bars to 1/32 with dotted and triplets, plus 1/64).

n1m21n/Infinite

NVIDIA DeepStream SDK development with Python pyservicemaker API.

NVIDIA/skills

A skill your agent uses when building, deploying, evaluating, debugging, or measuring latency for the DeepStream SOP Inference Microservice — a GPU-accelerated FastAPI service that detects whether…

NVIDIA/skills

Install or repair the FoundationPose perception pipeline and build its FoundationStereo TensorRT engines.

NVIDIA/skills

A skill your agent uses for NVIDIA-related requests where an NVIDIA skill might help, even if the user did not ask for a skill.

NVIDIA/skills

CLIP vision-language model for image-text retrieval, zero-shot classification, embedding extraction, ONNX export, and TensorRT deployment.

NVIDIA/skills

InternVideo2-CLIP L14 (TAO videoclip) for video-text retrieval, zero-shot classification, embedding extraction, LoRA fine-tuning, ONNX export, and TensorRT deployment.

NVIDIA/skills

The mandatory pre-launch gate and four-verb execution contract for every TAO workflow or action.

NVIDIA/skills

PyTorch-based TAO image classification. An agent skill from NVIDIA/skills.

NVIDIA/skills

RT-DETR (Real-Time DEtection TRansformer) for 2D object detection.

NVIDIA/skills

Run coding tasks via opencode using free cloud models. An agent skill from luongnv89/skills.

luongnv89/skills

Cross-engine decision rubric for self-hosting or recommending an LLM serving stack.

agentsope/SkillAlchemy

Project-kickoff rubric for the self-host vs managed-cloud decision — when is running your own inference engine / LLM platform worth the ops cost vs paying per-token for a managed API?

agentsope/SkillAlchemy

Decision SOP for serving LLMs with vLLM. An agent skill from agentsope/SkillAlchemy.

agentsope/SkillAlchemy

Converts one or more images into faithful text descriptions or OCR with the local 1.3B MiniCPM-V 4.6 GGUF model through llama.cpp, automatically preferring an available Vulkan GPU and falling back…

godot-fun/gai

Redact, anonymize, sanitize, or remove PII locally with Distil-PII and llama.cpp; keep personal data and secret values out of model context, logs, and chat.

HybridAIOne/hybridclaw

Reuse what Ollama already has on this computer: adopt a running Ollama server into the person's fleet, or serve an Ollama-downloaded model without Ollama.

autonomous-ai/openharness

Serve a Hugging Face model with vLLM on a Linux machine with an NVIDIA or AMD GPU, configured from the model's official vLLM recipe — or, when it has none, from the model's own files — and join it…

autonomous-ai/openharness

A Principal ML Engineer interviewer that simulates a FAANG-style ML system design interview covering the full lifecycle from data to production.

PrepLabsAI/InterviewMentor

Run quick, offline, private LLM tasks on local models via llama.cpp, reusing models already downloaded by Ollama.

glebis/claude-skills

Audit + recover a finished agentic eval. An agent skill from open-thoughts/OpenThoughts-Agent.

open-thoughts/OpenThoughts-Agent

Diagnose Day-2 AKS GPU and KAITO incidents using profile-aware, read-only evidence.

microsoft/GitHub-Copilot-for-Azure

Build a Quark ONNX PTQ quantization plan from modelanalysis.json and user intent.

amd/Quark

Validate Quark ONNX quantization output using four lightweight checks: auxiliary file copy alignment, expected non-quantized initializer MD5 byte-identity (inline rawdata + external-data byte…

amd/Quark

Route Quark ONNX user goals to the correct atomic skill. An agent skill from amd/Quark.

amd/Quark

Apply existing ShapeShifter graph passes to an .onnx model via the quark-cli shapeshifter CLI or a ShapeShifter YAML.

amd/Quark

Detect upstream Quark ONNX changes that affect the ONNX skill family and classify required updates.

amd/Quark

Partition an ONNX model graph into named functional subgraphs and emit a subgraphpartition.json file.

amd/Quark

Diagnose failed Quark installation, PTQ execution, script generation, or export attempts.

amd/Quark

Prepare export and downstream evaluation handoff for a planned or completed Quark PTQ run.

amd/Quark

Low-memory file2file quantization for very large safetensors LLMs that cannot be loaded whole.

amd/Quark

L3 recipe that runs a Torch LLM PTQ end-to-end for AMD Quark — for PyTorch / HuggingFace transformers models (safetensors input): quantize → validate → evaluate.

amd/Quark

Torch LLM PTQ workflow for AMD Quark — from model selection to quantized output.

amd/Quark

Inspect a target model and prepare metadata for Quark PTQ planning.

amd/Quark