Cache Policy Hit Rate Comparison
ben-manes/caffeine
Compares cache eviction policies by hit rate across several cache sizes on a trace file using the Caffeine simulator, with CSV tables and a PNG chart.
Compare and implement eviction policies (LRU, LFU, FIFO, S3FIFO, ARC) for bounded-capacity caches.
Apache-2.0Auto-check passed
Install Cache Policy Comparison
$ npx skills add benchflow-ai/skillsbench --skill cache-policy-comparison -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install benchflow-ai/skillsbench cache-policy-comparison --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/benchflow-ai/skillsbench.git skills-src && mkdir -p .claude/skills && cp -r skills-src/tasks/llm-prefix-cache-replay/environment/skills/cache-policy-comparison .claude/skills/cache-policy-comparison && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "cache-policy-comparison" agent skill from https://github.com/benchflow-ai/skillsbench/tree/main/tasks/llm-prefix-cache-replay/environment/skills/cache-policy-comparison into .claude/skills/cache-policy-comparison/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "cache-policy-comparison", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/benchflow-ai/skillsbench/tree/main/tasks/llm-prefix-cache-replay/environment/skills/cache-policy-comparisonType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add benchflow-ai/skillsbench --skill cache-policy-comparison -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install benchflow-ai/skillsbench cache-policy-comparison --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/benchflow-ai/skillsbench.git skills-src && mkdir -p .agents/skills && cp -r skills-src/tasks/llm-prefix-cache-replay/environment/skills/cache-policy-comparison .agents/skills/cache-policy-comparison && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "cache-policy-comparison" agent skill from https://github.com/benchflow-ai/skillsbench/tree/main/tasks/llm-prefix-cache-replay/environment/skills/cache-policy-comparison into .agents/skills/cache-policy-comparison/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "cache-policy-comparison", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add benchflow-ai/skillsbench --skill cache-policy-comparison -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install benchflow-ai/skillsbench cache-policy-comparison --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/benchflow-ai/skillsbench.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/tasks/llm-prefix-cache-replay/environment/skills/cache-policy-comparison .cursor/skills/cache-policy-comparison && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "cache-policy-comparison" agent skill from https://github.com/benchflow-ai/skillsbench/tree/main/tasks/llm-prefix-cache-replay/environment/skills/cache-policy-comparison into .cursor/skills/cache-policy-comparison/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "cache-policy-comparison", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/benchflow-ai/skillsbench.git --path tasks/llm-prefix-cache-replay/environment/skills/cache-policy-comparison--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add benchflow-ai/skillsbench --skill cache-policy-comparison -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install benchflow-ai/skillsbench cache-policy-comparison --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/benchflow-ai/skillsbench.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/tasks/llm-prefix-cache-replay/environment/skills/cache-policy-comparison .gemini/skills/cache-policy-comparison && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "cache-policy-comparison" agent skill from https://github.com/benchflow-ai/skillsbench/tree/main/tasks/llm-prefix-cache-replay/environment/skills/cache-policy-comparison into .gemini/skills/cache-policy-comparison/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "cache-policy-comparison", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install benchflow-ai/skillsbench cache-policy-comparisonInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add benchflow-ai/skillsbench --skill cache-policy-comparison -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/benchflow-ai/skillsbench.git skills-src && mkdir -p .github/skills && cp -r skills-src/tasks/llm-prefix-cache-replay/environment/skills/cache-policy-comparison .github/skills/cache-policy-comparison && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "cache-policy-comparison" agent skill from https://github.com/benchflow-ai/skillsbench/tree/main/tasks/llm-prefix-cache-replay/environment/skills/cache-policy-comparison into .github/skills/cache-policy-comparison/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "cache-policy-comparison", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add benchflow-ai/skillsbench --skill cache-policy-comparison -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install benchflow-ai/skillsbench cache-policy-comparison --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/benchflow-ai/skillsbench.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/tasks/llm-prefix-cache-replay/environment/skills/cache-policy-comparison .opencode/skills/cache-policy-comparison && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "cache-policy-comparison" agent skill from https://github.com/benchflow-ai/skillsbench/tree/main/tasks/llm-prefix-cache-replay/environment/skills/cache-policy-comparison into .opencode/skills/cache-policy-comparison/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "cache-policy-comparison", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
Facts
- Skill name
cache-policy-comparison- GitHub stars
- 1.8k
- Token cost
- ~1.5k tokens
- SKILL.md length
- 642 words
- Files
- 1
- Skills in repo
- 178
- Repo updated
- First seen
- Licence
- Apache-2.0
At a glance
Compare and implement eviction policies (LRU, LFU, FIFO, S3FIFO, ARC) for bounded-capacity caches.
- Implementing an eviction policy for a buffer pool
- Instructions only: no scripts, shell commands, URLs or credentials in SKILL.md
- Writing a replay simulator that supports multiple policies
What it does
Cache Policy Comparison is an agent skill from benchflow-ai/skillsbench. Compare and implement eviction policies (LRU, LFU, FIFO, S3FIFO, ARC) for bounded-capacity caches. Use when choosing or implementing an eviction policy for a buffer pool, page cache, CDN edge, or LLM KV cache, or when writing a replay simulator that supports multiple policies. Clarifies recency vs frequency semantics, queue topology, saturating counters, ghost buffers, and the second-chance rule that distinguishes modern FIFO-family policies from classic LRU.
Its SKILL.md is about 1.5k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.
The repository describes itself as: SkillsBench evaluates how well skills work and how effective agents are at using them. The licence is Apache-2.0.
When your agent uses it
- Implementing an eviction policy for a buffer pool
- Writing a replay simulator that supports multiple policies
Example prompts
- “/cache-policy-comparison”
Requirements
- Python 3
What it can do on your machine
Read from SKILL.md and the folder at commit 9a1f4dd. It shows what the files ask for, not the result of running them.
Tool permissions
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Runs code
No scripts in the folder and no shell commands in SKILL.md (its code samples are python).
From the folder's file list and the shell code blocks in SKILL.md.
Network
No URLs in SKILL.md.
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Context cost
Cache Policy Comparison loads about 1.5k tokens when it runs. Until then it costs about 122 tokens; SKILL.md has 642 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
Safety
Auto-check passed
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.
SKILL.md
The full file from benchflow-ai/skillsbench at commit 9a1f4dd, republished under its Apache-2.0 licence (© benchflow-ai). 642 words, ~1,485 tokens.
Download SKILL.mdSave it as
.claude/skills/cache-policy-comparison/SKILL.md (or your agent's skills folder).- name
- cache-policy-comparison
- description
- Compare and implement eviction policies (LRU, LFU, FIFO, S3FIFO, ARC) for bounded-capacity caches. Use when choosing or implementing an eviction policy for a buffer pool, page cache, CDN edge, or LLM KV cache, or when writing a replay simulator that supports multiple policies. Clarifies recency vs frequency semantics, queue topology, saturating counters, ghost buffers, and the second-chance rule that distinguishes modern FIFO-family policies from classic LRU.
Overview
An eviction policy decides which resident entry a cache removes when a new entry is admitted beyond capacity. Four policies cover almost every replay-and-measure task:
| Policy | Data structure | On hit | On admit | Eviction choice |
|---|---|---|---|---|
| LRU | OrderedDict | Move to tail | Append at tail | Pop head |
| LFU | {key: freq} + insertion order | freq[k] += 1 | freq[k] = 1 | Min freq, tiebreak by insertion order |
| FIFO | OrderedDict | Nothing | Append at tail | Pop head |
| S3FIFO | Three FIFO queues + freq[k] | freq[k] = min(freq+1, cap) | Admit to small; ghost-hit admits to main | Second-chance on main; small drains to main/ghost |
Each has subtleties that trip naive implementations.
LRU
Use an OrderedDict where the tail is the most-recently-accessed key. On hit, move_to_end. On miss + insert, append; pop from head if over capacity.
Most common bug: forgetting to update recency on a hit. Without the refresh, LRU degenerates to FIFO — hit rate drops substantially on any workload with recency structure.
from collections import OrderedDict
class LRU:
def __init__(self, capacity):
self.capacity = capacity
self._d = OrderedDict()
def contains(self, k): return k in self._d
def access(self, k):
if k in self._d:
self._d.move_to_end(k)
else:
self._d[k] = None
if len(self._d) > self.capacity:
self._d.popitem(last=False)LFU
Keep freq: dict[key, int] and a tie-breaker — an insertion counter is simplest and deterministic. On hit, increment freq[k]. On miss at capacity, evict min(freq) with ties broken by insertion order (oldest first).
Typical bugs:
- No tie-breaker.
min(freq.items(), key=lambda x: x[1])[0]has implementation-defined behaviour across interpreters and distributions. Always include a secondary key. - Frequency pollution. A block that was hot once and then went cold can linger forever because its freq is permanently above newcomers. Production systems add aging (periodic decay of freq) or combine with a recency signal (W-TinyLFU). Pure LFU is correct for the task as specified but fragile in practice.
FIFO
One queue, insertion order, no hit-time update. Useful as a lower-bound baseline.
Do NOT call it "LRU without hit update" — conceptually different even when implementations overlap. Hit on a FIFO cache is still a hit for accounting; the block just does not change rank.
S3FIFO
A modern FIFO-family policy (Yang et al., SOSP 2023) that matches or beats LRU on typical web and LLM workloads with a fraction of the bookkeeping cost — which is why recent production systems (Twitter, Google) have been switching to it. The full algorithm — three queues, saturating frequency counter, second-chance eviction on the main queue — is implemented in the prefix-cache-replay skill. Consult that skill if your task uses S3FIFO.
Show full SKILL.md (277 more words)Show less
Workload implications
- Strong recency → LRU wins slightly.
- Stable hot set with long tail (Zipf) → LFU or S3FIFO.
- Nearly uniform random → all converge toward
capacity / working_sethit rate. - Prefix-shared LLM workloads are mixed — shared prefixes are both recent and frequent, so LRU/LFU/S3FIFO typically sit within a few percent of each other at the same capacity, but they differ in which blocks remain resident at end-of-trace, and their miss-handling costs diverge. Measure, don't assume.
Comparing hit rates on a trace
Replay the same trace through each policy at identical capacity, record total_hit_tokens / total_prompt_tokens and the final resident set. Do not compare hit rate alone — also compare:
- Final residency — how many unique blocks are resident at the end. Under S3FIFO this is often strictly less than capacity because ghost entries absorb the admission pressure.
- Per-request hit-token distribution — two policies can have similar overall hit rate but very different per-request variance.
- Admission effort — under policies with ghost structures, the bookkeeping cost per access is non-trivial.
Common mistakes
- Reusing an LRU implementation when the task specifies S3FIFO (or vice versa). The final hit rate and residency will both differ; no partial credit for "close enough".
- Making ghost count as resident, or treating a ghost hit as a hit for token accounting.
- Forgetting to saturate
freq— unbounded counters turn the main-queue second-chance loop into a spin. - Under LFU, using Python
min(d.items(), key=d.get)without an explicit insertion-order tiebreaker. - Misordering admission and residency check. Always check
h ∈ cacheBEFORE applying the admission side effects of the current request, otherwise every request self-hits. - Final cache size off by small constants because you forgot to exclude ghost or you forgot to subtract the S-cap vs M-cap split.
© benchflow-ai, Apache-2.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
Files
Just SKILL.md in tasks/llm-prefix-cache-replay/environment/skills/cache-policy-comparison of benchflow-ai/skillsbench.
Open the folder on GitHubat commit 9a1f4dd
Compare with similar skills
Cache Policy Comparison next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Cache Policy Comparison this skillbenchflow-ai/skillsbench | 1.8k | — | ~1.5k | Automated safety check: Pass | Apache-2.0 | |
| Cache Policy Hit Rate Comparisonben-manes/caffeine | 18k | — | ~939 | Automated safety check: Notes | Apache-2.0 | |
| Prompt Cachingdavila7/claude-code-templates | 32k | 6 repos | ~452 | Automated safety check: Pass | MIT | |
| Turborepo Cachingwshobson/agents | 40k | 9 repos | ~2k | Automated safety check: Notes | MIT | |
| OmniRoute LLM Cachediegosouzapw/OmniRoute | 74k | 1 repos | ~529 | Automated safety check: Pass | MIT | |
| Cachingzebbern/claude-code-guide | 4.7k | — | ~1.5k | Automated safety check: Pass | MIT |
Similar skills
Prompt Caching
davila7/claude-code-templates
Caching strategies for LLM prompts including Anthropic prompt caching, response caching, and CAG (Cache Augmented Generation) Use when: prompt caching, cache prompt, response cache, cag, cache…
Turborepo Caching
wshobson/agents
Configures Turborepo pipelines and local or remote caching for monorepo builds, including Vercel remote cache, a self-hosted cache and cache-miss debugging.
OmniRoute LLM Cache
diegosouzapw/OmniRoute
Documents OmniRoute's cache endpoints for reading cache statistics and clearing entries, statistics or the reasoning cache, with notes on TTL and similarity settings.
Caching
zebbern/claude-code-guide
Caching strategies — invalidation, TTL guidelines, cache keys, cache layers, and when not to cache.
Implementing Policy As Code With Open Policy Agent
mukul975/Anthropic-Cybersecurity-Skills
Implements policy-as-code enforcement with Open Policy Agent (OPA) and Gatekeeper for Kubernetes and CI/CD pipelines, covering writing Rego policies, deploying OPA Gatekeeper as a Kubernetes…
More from benchflow-ai/skillsbench
All 178 skills in this repoLean4 Memories
benchflow-ai/skillsbench
This skill should be used when working on Lean 4 formalization projects to maintain persistent memory of successful proof patterns, failed approaches, project conventions, and user preferences…
Senior Data Engineer
benchflow-ai/skillsbench
World-class data engineering skill for building scalable data pipelines, ETL/ELT systems, real-time streaming, and data infrastructure.
Ac Branch Pi Model
benchflow-ai/skillsbench
AC branch pi-model power flow equations (P/Q and |S|) with transformer tap ratio and phase shift, matching acopf-math-model.md and MATPOWER branch fields.
Civ6lib
benchflow-ai/skillsbench
Civilization 6 district mechanics library. An agent skill from benchflow-ai/skillsbench.
D3 Visualization
benchflow-ai/skillsbench
Build deterministic, verifiable data visualizations with D3.js (v6).
Dc Power Flow
benchflow-ai/skillsbench
DC power flow analysis for power systems. An agent skill from benchflow-ai/skillsbench.
Questions about Cache Policy Comparison
What does Cache Policy Comparison do?
Compare and implement eviction policies (LRU, LFU, FIFO, S3FIFO, ARC) for bounded-capacity caches. Cache Policy Comparison is an agent skill from benchflow-ai/skillsbench. Compare and implement eviction policies (LRU, LFU, FIFO, S3FIFO, ARC) for bounded-capacity caches.
When should I use Cache Policy Comparison?
Cache Policy Comparison fits situations like: implementing an eviction policy for a buffer pool; writing a replay simulator that supports multiple policies.
How do I install Cache Policy Comparison in Claude Code?
Run `npx skills add benchflow-ai/skillsbench --skill cache-policy-comparison -a claude-code`. Or copy the skill folder (tasks/llm-prefix-cache-replay/environment/skills/cache-policy-comparison in benchflow-ai/skillsbench) into .claude/skills/cache-policy-comparison in your project. Claude Code loads it when a task matches its description.
How do I install Cache Policy Comparison in Codex?
Run `npx skills add benchflow-ai/skillsbench --skill cache-policy-comparison -a codex`. Or copy the skill folder (tasks/llm-prefix-cache-replay/environment/skills/cache-policy-comparison in benchflow-ai/skillsbench) into .agents/skills/cache-policy-comparison in your project. Codex loads it when a task matches its description.
Can I use Cache Policy Comparison in Cursor, Gemini CLI or GitHub Copilot?
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add benchflow-ai/skillsbench --skill cache-policy-comparison -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/cache-policy-comparison, .gemini/skills/cache-policy-comparison, .github/skills/cache-policy-comparison and .opencode/skills/cache-policy-comparison in your project.
What does Cache Policy Comparison need to run?
SKILL.md names no scripts, command-line tools or credentials: Cache Policy Comparison is instructions for the agent only. Our summary lists: Python 3.
Does Cache Policy Comparison access the network?
SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.
Is Cache Policy Comparison safe to install?
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.
What licence does Cache Policy Comparison use?
Cache Policy Comparison is published under the Apache-2.0 licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.
How many tokens does Cache Policy Comparison use?
About 1.5k tokens (SKILL.md is roughly 5.9k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.
What are the alternatives to Cache Policy Comparison?
Skills that share tags, products or a category with Cache Policy Comparison: Cache Policy Hit Rate Comparison (ben-manes/caffeine, 18k stars), Prompt Caching (davila7/claude-code-templates, 32k stars), Turborepo Caching (wshobson/agents, 40k stars) and OmniRoute LLM Cache (diegosouzapw/OmniRoute, 74k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
Who maintains Cache Policy Comparison?
benchflow-ai (a GitHub organization) maintains it in benchflow-ai/skillsbench, which has 1,832 GitHub stars. The repository holds 178 skills in this directory. The repository was last updated on July 23, 2026.
Source: benchflow-ai/skillsbench on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.