Groq Performance Tuning

What it does

When your agent uses it

• Experiencing slow responses
• Implementing caching strategies
• Optimizing request throughput for Groq integrations
• With phrases like groq performance

Example prompts

• “groq performance”
• “optimize groq”
• “groq latency”
• “/groq-performance-tuning”

Requirements

• Node.js
• A credential in GROQ_API_KEY
• Compatibility (from SKILL.md): Designed for Claude Code
• Pre-approved tools (allowed-tools): Read, Write, Edit

Workflow steps

6 steps, taken from the first numbered list in SKILL.md.

1. 1Choose the right model for speed. Map each call site to a speed tier: llama-3.1-8b-instant for latency-critical paths…
2. 2Minimize token count. Trim verbose system prompts to their essence and set max_tokens to the expected output size, not a safe-looking…
3. 3Stream for perceived performance. For any output the user watches arrive, stream chunks and surface live TTFT / tokens-per-second metrics…
4. 4Cache deterministic responses. Hash {messages, model} and serve repeat temperature: 0 requests from an LRU cache with a short TTL…
5. 5Parallelize under a rate-limit-aware queue. Fan out bulk work with p-queue, capping concurrency and per-minute volume so you saturate…
6. 6Benchmark before you commit. Measure the candidate models against your real prompt shape and pick the fastest that clears your quality bar.

What it can do on your machine

Read from SKILL.md and the folder at commit cfae287. It shows what the files ask for, not the result of running them.

• Tool permissions

  Pre-approves these tools, so the agent can use them without asking each time:

  • Read
  • Write
  • Edit

  From allowed-tools in the SKILL.md frontmatter.

• Runs code

  Shell commands in SKILL.md call:

  • npm

  From the folder's file list and the shell code blocks in SKILL.md.

• Network

  Links to these hosts (documentation or services it may open):

  • console.groq.com
  • npmjs.com

  From URLs in SKILL.md, links to its own repository left out.

• Credentials

  Names these keys or tokens, usually read from environment variables:

  • GROQ_API_KEY

  From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

• Compatibility

  Designed for Claude Code

  From compatibility in the SKILL.md frontmatter.

Context cost

Groq Performance Tuning loads about 1.9k tokens when it runs, and up to ~4k if it reads all its reference files. Until then it costs about 92 tokens; SKILL.md has 711 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it

~92

When it runs · the whole SKILL.md, loaded when a task matches

~1.9k

With references · SKILL.md plus every file in references/, read only if the agent opens them

~4k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from jeremylongshore/tons-of-skills-marketplace at commit cfae287, republished under its MIT licence (© jeremylongshore). 711 words, ~1,950 tokens.

name

groq-performance-tuning

description

Optimize Groq API performance with model selection, caching, streaming, and parallel requests. Use when experiencing slow responses, implementing caching strategies, or optimizing request throughput for Groq integrations. Trigger with phrases like "groq performance", "optimize groq", "groq latency", "groq caching", "groq slow", "groq speed".

allowed-tools

Read, Write, Edit

compatibility

Designed for Claude Code

version

1.11.0

license

MIT

author

Jeremy Longshore <jeremy@intentsolutions.io>

tags

saas, groq, api, performance

import Groq from "groq-sdk";

const groq = new Groq();  // reads GROQ_API_KEY from the environment

const SPEED_MAP = {
  instant: "llama-3.1-8b-instant",      // <100ms TTFT — latency-critical
  balanced: "llama-3.3-70b-versatile",  // <200ms TTFT — quality-sensitive
  fast70b: "llama-3.3-70b-specdec",     // 70b quality, faster throughput
} as const;

async function tieredCompletion(prompt: string, tier: keyof typeof SPEED_MAP = "instant") {
  return groq.chat.completions.create({
    model: SPEED_MAP[tier],
    messages: [{ role: "user", content: prompt }],
    temperature: 0,   // deterministic = cacheable
    max_tokens: 256,  // request only what you need
  });
}

llama-3.1-8b-instant     |  61ms avg | 548 tok/s avg
llama-3.3-70b-versatile  | 148ms avg | 279 tok/s avg
llama-3.3-70b-specdec    | 103ms avg | 401 tok/s avg

// Latency-critical classification, cached
const label = await cachedCompletion(
  [
    { role: "system", content: "Classify as positive/negative/neutral. One word only." },
    { role: "user", content: "This product exceeded every expectation." },
  ],
  "llama-3.1-8b-instant"
);
// => "positive"

© jeremylongshore, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 2 other files (references) in skills/.curated/groq-performance-tuning of jeremylongshore/tons-of-skills-marketplace.

• SKILL.md
• references/examples.md
• references/implementation.md

Open the folder on GitHubat commit cfae287