Glean Rate Limits

What it does

When your agent uses it

• Tasks that involve Rate limiting

Example prompts

• “glean rate limits”
• “rate-limits”
• “/glean-rate-limits”

Requirements

• Compatibility (from SKILL.md): Designed for Claude Code
• Pre-approved tools (allowed-tools): Read, Write, Edit, Bash(npm:*), Bash(curl:*), Grep

Workflow steps

5 steps, taken from the first numbered list in SKILL.md.

1. 1Classify calls as read or write, apply per-scope concurrency limits, and honor server retry guidance when it is present.
2. 2Use bounded exponential backoff with jitter and a maximum attempt count; never blindly replay a write without its idempotency key.
3. 3Shed or defer nonessential work before queue growth threatens freshness or downstream access-control synchronization.
4. 4Monitor aggregate rate-limit responses and canary success, then tune one scope at a time with a rollback to the prior limits.
5. 5Route exhausted items to reviewed recovery rather than silently dropping or duplicating them.

What it can do on your machine

Read from SKILL.md and the folder at commit cfae287. It shows what the files ask for, not the result of running them.

• Tool permissions

  Pre-approves these tools, so the agent can use them without asking each time:

  • Read
  • Write
  • Edit
  • Bash(npm:*)
  • Bash(curl:*)
  • Grep

  From allowed-tools in the SKILL.md frontmatter.

• Runs code

  No scripts in the folder and no shell commands in SKILL.md (its code samples are typescript).

  From the folder's file list and the shell code blocks in SKILL.md.

• Network

  Links to these hosts (documentation or services it may open):

  • developers.glean.com

  From URLs in SKILL.md, links to its own repository left out.

• Credentials

  Names no API keys, tokens, secrets or passwords.

  From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

• Compatibility

  Designed for Claude Code

  From compatibility in the SKILL.md frontmatter.

Context cost

Glean Rate Limits loads about 1.5k tokens when it runs. Until then it costs about 28 tokens; SKILL.md has 401 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it

~28

When it runs · the whole SKILL.md, loaded when a task matches

~1.5k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from jeremylongshore/tons-of-skills-marketplace at commit cfae287, republished under its MIT licence (© jeremylongshore). 401 words, ~1,497 tokens.

name

glean-rate-limits

description

Glean Indexing API: ~100 requests/min per token. Trigger: "glean rate limits", "rate-limits".

allowed-tools

Read, Write, Edit, Bash(npm:*), Bash(curl:*), Grep

compatibility

Designed for Claude Code

version

1.8.0

license

MIT

author

Jeremy Longshore <jeremy@intentsolutions.io>

tags

saas, enterprise-search, glean

class GleanRateLimiter {
  private tokens: number;
  private lastRefill: number;
  private readonly max: number;
  private readonly refillRate: number;
  private queue: Array<{ resolve: () => void }> = [];

  constructor(maxPerMinute: number) {
    this.max = maxPerMinute;
    this.tokens = maxPerMinute;
    this.lastRefill = Date.now();
    this.refillRate = maxPerMinute / 60_000;
  }

  async acquire(): Promise<void> {
    this.refill();
    if (this.tokens >= 1) { this.tokens -= 1; return; }
    return new Promise(resolve => this.queue.push({ resolve }));
  }

  private refill() {
    const now = Date.now();
    this.tokens = Math.min(this.max, this.tokens + (now - this.lastRefill) * this.refillRate);
    this.lastRefill = now;
    while (this.tokens >= 1 && this.queue.length) {
      this.tokens -= 1;
      this.queue.shift()!.resolve();
    }
  }
}

const indexLimiter = new GleanRateLimiter(18);  // buffer under 20 bulk/min
const searchLimiter = new GleanRateLimiter(50);

async function gleanRetry<T>(
  limiter: GleanRateLimiter, fn: () => Promise<Response>, maxRetries = 4
): Promise<T> {
  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    await limiter.acquire();
    const res = await fn();
    if (res.ok) return res.json();
    if (res.status === 429) {
      const retryAfter = parseInt(res.headers.get("Retry-After") || "30", 10);
      const jitter = Math.random() * 5000;
      await new Promise(r => setTimeout(r, retryAfter * 1000 + jitter));
      continue;
    }
    if (res.status >= 500 && attempt < maxRetries) {
      await new Promise(r => setTimeout(r, Math.pow(2, attempt) * 2000));
      continue;
    }
    throw new Error(`Glean API ${res.status}: ${await res.text()}`);
  }
  throw new Error("Max retries exceeded");
}

async function bulkIndexDocuments(docs: any[], batchSize = 100) {
  const results: any[] = [];
  for (let i = 0; i < docs.length; i += batchSize) {
    const batch = docs.slice(i, i + batchSize);
    const result = await gleanRetry(indexLimiter, () =>
      fetch(`${GLEAN_BASE}/api/index/v1/bulkindexdocuments`, {
        method: "POST", headers,
        body: JSON.stringify({ uploadId: `batch-${i}`, documents: batch }),
      })
    );
    results.push(result);
    if (i + batchSize < docs.length) await new Promise(r => setTimeout(r, 4000));
  }
  return results;
}

© jeremylongshore, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in skills/.curated/glean-rate-limits of jeremylongshore/tons-of-skills-marketplace.

Open the folder on GitHubat commit cfae287