LLM Torch Profiler Analysis
sgl-project/sglang
Unified LLM torch-profiler triage skill for sglang, vllm, TensorRT-LLM, and TokenSpeed.
Optimize FlashAttention-style fused attention kernels in Triton for NVIDIA and AMD GPUs.
No licenceAuto-check passed
Install Flash Attention Kernel
$ npx skills add ZJLi2013/awesome-kernel-skills --skill flash-attention-kernel -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install ZJLi2013/awesome-kernel-skills flash-attention-kernel --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/ZJLi2013/awesome-kernel-skills.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/kernels/flash-attention .claude/skills/flash-attention-kernel && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "flash-attention-kernel" agent skill from https://github.com/ZJLi2013/awesome-kernel-skills/tree/main/skills/kernels/flash-attention into .claude/skills/flash-attention-kernel/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "flash-attention-kernel", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/ZJLi2013/awesome-kernel-skills/tree/main/skills/kernels/flash-attentionType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add ZJLi2013/awesome-kernel-skills --skill flash-attention-kernel -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install ZJLi2013/awesome-kernel-skills flash-attention-kernel --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/ZJLi2013/awesome-kernel-skills.git skills-src && mkdir -p .agents/skills && cp -r skills-src/skills/kernels/flash-attention .agents/skills/flash-attention-kernel && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "flash-attention-kernel" agent skill from https://github.com/ZJLi2013/awesome-kernel-skills/tree/main/skills/kernels/flash-attention into .agents/skills/flash-attention-kernel/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "flash-attention-kernel", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add ZJLi2013/awesome-kernel-skills --skill flash-attention-kernel -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install ZJLi2013/awesome-kernel-skills flash-attention-kernel --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/ZJLi2013/awesome-kernel-skills.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/skills/kernels/flash-attention .cursor/skills/flash-attention-kernel && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "flash-attention-kernel" agent skill from https://github.com/ZJLi2013/awesome-kernel-skills/tree/main/skills/kernels/flash-attention into .cursor/skills/flash-attention-kernel/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "flash-attention-kernel", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/ZJLi2013/awesome-kernel-skills.git --path skills/kernels/flash-attention--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add ZJLi2013/awesome-kernel-skills --skill flash-attention-kernel -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install ZJLi2013/awesome-kernel-skills flash-attention-kernel --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/ZJLi2013/awesome-kernel-skills.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/skills/kernels/flash-attention .gemini/skills/flash-attention-kernel && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "flash-attention-kernel" agent skill from https://github.com/ZJLi2013/awesome-kernel-skills/tree/main/skills/kernels/flash-attention into .gemini/skills/flash-attention-kernel/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "flash-attention-kernel", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install ZJLi2013/awesome-kernel-skills flash-attention-kernelInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add ZJLi2013/awesome-kernel-skills --skill flash-attention-kernel -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/ZJLi2013/awesome-kernel-skills.git skills-src && mkdir -p .github/skills && cp -r skills-src/skills/kernels/flash-attention .github/skills/flash-attention-kernel && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "flash-attention-kernel" agent skill from https://github.com/ZJLi2013/awesome-kernel-skills/tree/main/skills/kernels/flash-attention into .github/skills/flash-attention-kernel/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "flash-attention-kernel", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add ZJLi2013/awesome-kernel-skills --skill flash-attention-kernel -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install ZJLi2013/awesome-kernel-skills flash-attention-kernel --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/ZJLi2013/awesome-kernel-skills.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/skills/kernels/flash-attention .opencode/skills/flash-attention-kernel && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "flash-attention-kernel" agent skill from https://github.com/ZJLi2013/awesome-kernel-skills/tree/main/skills/kernels/flash-attention into .opencode/skills/flash-attention-kernel/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "flash-attention-kernel", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
Facts
- Skill name
flash-attention-kernel- GitHub stars
- 102
- Token cost
- ~697 tokens
- SKILL.md length
- 302 words
- Files
- 3
- Skills in repo
- 12
- Repo updated
- First seen
- Licence
- None found
At a glance
Optimize FlashAttention-style fused attention kernels in Triton for NVIDIA and AMD GPUs.
- Works in 3 steps: Maintain running m_i (row max) and l_i… → For each K tile: compute qk = Q @ K^T,… → After all K tiles: divide accumulator by…
- Optimizing self-attention
- SKILL.md covers Overview, Core Technique, Autotune Notes and Verification, plus 2 more sections
- Runs Python scripts from its folder; calls python
What it does
Flash Attention Kernel is an agent skill from ZJLi2013/awesome-kernel-skills. Optimize FlashAttention-style fused attention kernels in Triton for NVIDIA and AMD GPUs. Covers online softmax, tiled QK/AV GEMM, causal masking, and memory-efficient attention. Use when writing or optimizing self-attention, cross-attention, or any QKV attention kernel.
Its SKILL.md is about 700 tokens, which your agent loads only when the skill is triggered. The skill folder holds 2 other files (for example `test_flash_attention.py` and `triton_template.py`).
It works with NVIDIA AI Platform.
When your agent uses it
- Optimizing self-attention
- Cross-attention
- Any QKV attention kernel
Example prompts
- “/flash-attention-kernel”
Requirements
- Python 3
Workflow steps
3 steps, taken from the first numbered list in SKILL.md.
- Maintain running m_i (row max) and l_i (exp sum) per query row
- For each K tile: compute qk = Q @ K^T, update max, rescale accumulator
- After all K tiles: divide accumulator by final l_i
What it can do on your machine
Read from SKILL.md and the folder at commit aba7662. It shows what the files ask for, not the result of running them.
Tool permissions
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Runs code
Ships script files (Python), which the agent can run.
Shell commands in SKILL.md call:
python
From the folder's file list and the shell code blocks in SKILL.md.
Network
Links to these hosts (documentation or services it may open):
arxiv.orgtriton-lang.org
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Context cost
Flash Attention Kernel loads about 697 tokens when it runs. Until then it costs about 73 tokens; SKILL.md has 302 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
Safety
Auto-check passed
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.
SKILL.md
Without a licence we can't republish the file, so here is its outline and opening line. It has 302 words (~697 tokens).
“FlashAttention fuses the entire attention computation (Q@K^T -> softmax -> @V) into a single kernel, avoiding materialization of the full N x N attention matrix in HBM.”
- name
- flash-attention-kernel
Files
SKILL.md and 2 other files in skills/kernels/flash-attention of ZJLi2013/awesome-kernel-skills.
- SKILL.md
- test_flash_attention.py
- triton_template.py
Open the folder on GitHubat commit aba7662
Compare with similar skills
Flash Attention Kernel next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Flash Attention Kernel this skillZJLi2013/awesome-kernel-skills | 102 | — | ~697 | Automated safety check: Pass | None | |
| LLM Torch Profiler Analysissgl-project/sglang | 37k | 2 repos | ~6.4k | Automated safety check: Pass | Apache-2.0 | |
| Skill InspectorNVIDIA/SkillSpector | 20k | — | ~1.8k | Automated safety check: Pass | Apache-2.0 | |
| Megatron-LM Container and Dependency SetupNVIDIA/Megatron-LM | 18k | — | ~2.6k | Automated safety check: Pass | Apache-2.0 | |
| Embeddings via 9Routerdecolua/9router | 30k | — | ~604 | Automated safety check: Pass | MIT | |
| Megatron-LM Base Image BumpNVIDIA/Megatron-LM | 18k | — | ~2.8k | Automated safety check: Pass | Apache-2.0 |
Similar skills
- Official
Skill Inspector
NVIDIA/SkillSpector
Decides whether an agent skill is safe to install by combining a SkillSpector static scan with the agent's own source review, ending in APPROVE, CAUTION or REJECT.
- Official
Megatron-LM Container and Dependency Setup
NVIDIA/Megatron-LM
Walks an agent through working inside the Megatron-LM CI container and changing dependencies with uv, so lock files resolve the same locally and in CI.
Embeddings via 9Router
decolua/9router
Generates vector embeddings through the 9Router /v1/embeddings endpoint, using models from providers such as OpenAI, Gemini, Mistral and Voyage for RAG and semantic search.
- Official
Megatron-LM Base Image Bump
NVIDIA/Megatron-LM
Moves Megatron-LM CI to a newer NVIDIA PyTorch base image, updating both the GitHub and GitLab pins together and handling the CI follow-up.
- Official
Nemoclaw Maintainer Normalize Title Tags
NVIDIA/NemoClaw
Remove bracketed NemoClaw tags from GitHub issue and PR titles.
More from ZJLi2013/awesome-kernel-skills
All 12 skills in this repoCross Entropy Kernel
ZJLi2013/awesome-kernel-skills
Optimize fused cross-entropy loss kernels in Triton for NVIDIA and AMD GPUs.
Fused Moe Kernel
ZJLi2013/awesome-kernel-skills
Optimize Fused Mixture-of-Experts (MoE) kernels in Triton for NVIDIA and AMD GPUs.
Gemm Kernel Optimization
ZJLi2013/awesome-kernel-skills
Optimize dense matrix multiplication (GEMM) kernels in Triton for NVIDIA and AMD GPUs.
Iterative Kernel Optimization Loop
ZJLi2013/awesome-kernel-skills
Orchestrates continuous kernel optimization by chaining profiling, bottleneck diagnosis, tier-based optimization, verification, and benchmarking into an iterative loop.
Kernel Benchmark
ZJLi2013/awesome-kernel-skills
Unified kernel benchmarking protocol producing JSON results with latency, TFLOPS, GBps, and comparison against PyTorch baselines.
Kernel Profiling
ZJLi2013/awesome-kernel-skills
Profile GPU kernels using NCU (NVIDIA) or rocprof (AMD) to collect performance metrics.
Related
Works with
Questions about Flash Attention Kernel
What does Flash Attention Kernel do?
Optimize FlashAttention-style fused attention kernels in Triton for NVIDIA and AMD GPUs. Flash Attention Kernel is an agent skill from ZJLi2013/awesome-kernel-skills. Optimize FlashAttention-style fused attention kernels in Triton for NVIDIA and AMD GPUs.
When should I use Flash Attention Kernel?
Flash Attention Kernel fits situations like: optimizing self-attention; cross-attention; any QKV attention kernel.
How do I install Flash Attention Kernel in Claude Code?
Run `npx skills add ZJLi2013/awesome-kernel-skills --skill flash-attention-kernel -a claude-code`. Or copy the skill folder (skills/kernels/flash-attention in ZJLi2013/awesome-kernel-skills) into .claude/skills/flash-attention-kernel in your project. Claude Code loads it when a task matches its description.
How do I install Flash Attention Kernel in Codex?
Run `npx skills add ZJLi2013/awesome-kernel-skills --skill flash-attention-kernel -a codex`. Or copy the skill folder (skills/kernels/flash-attention in ZJLi2013/awesome-kernel-skills) into .agents/skills/flash-attention-kernel in your project. Codex loads it when a task matches its description.
Can I use Flash Attention Kernel in Cursor, Gemini CLI or GitHub Copilot?
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add ZJLi2013/awesome-kernel-skills --skill flash-attention-kernel -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/flash-attention-kernel, .gemini/skills/flash-attention-kernel, .github/skills/flash-attention-kernel and .opencode/skills/flash-attention-kernel in your project.
What does Flash Attention Kernel need to run?
Going by SKILL.md and its folder, Flash Attention Kernel needs Python for the scripts in its folder and the command-line tools its instructions call (python). Our summary lists: Python 3.
Does Flash Attention Kernel access the network?
SKILL.md names 2 domains. As links in the text: arxiv.org and triton-lang.org. This is read from the text; nothing was executed.
Is Flash Attention Kernel safe to install?
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.
What licence does Flash Attention Kernel use?
No licence was found for Flash Attention Kernel or its repository. Without one, default copyright applies: ask the author before reusing or redistributing it.
How many tokens does Flash Attention Kernel use?
About 697 tokens (SKILL.md is roughly 2.8k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.
What are the alternatives to Flash Attention Kernel?
Skills that share tags, products or a category with Flash Attention Kernel: LLM Torch Profiler Analysis (sgl-project/sglang, 37k stars), Skill Inspector (NVIDIA/SkillSpector, 20k stars), Megatron-LM Container and Dependency Setup (NVIDIA/Megatron-LM, 18k stars) and Embeddings via 9Router (decolua/9router, 30k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
Who maintains Flash Attention Kernel?
ZJLi2013 (a GitHub user) maintains it in ZJLi2013/awesome-kernel-skills, which has 102 GitHub stars. The repository holds 12 skills in this directory. The repository was last updated on March 31, 2026.
Source: ZJLi2013/awesome-kernel-skills on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.