Verbalized Sampling

What it does

When your agent uses it

• The task needs genuine diversity: creative writing
• Brainstorming/ideation
• Synthetic data generation
• Persona/dialogue simulation

Example prompts

• “generate 5 ideas”
• “/verbalized-sampling”

Requirements

• Python 3

Workflow steps

4 steps, taken from the first numbered list in SKILL.md.

1. 1Is this naive/obvious? (would it appear in a top-10 listicle?)
2. 2Is this actionable? (could execution start Monday without further research?)
3. 3Does this require magical thinking? (assumes steps will "just work" with no mechanism)
4. 4Does this ignore base rates? (approaches with known low success rates presented as good bets)

What it can do on your machine

Read from SKILL.md and the folder at commit 43a0566. It shows what the files ask for, not the result of running them.

• Tool permissions

  Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

  From allowed-tools in the SKILL.md frontmatter.

• Runs code

  Ships 1 file in scripts/ (Python), which the agent can run.

  Shell commands in SKILL.md call:

  • python

  From the folder's file list and the shell code blocks in SKILL.md.

• Network

  No URLs in SKILL.md.

  From URLs in SKILL.md, links to its own repository left out.

• Credentials

  Names no API keys, tokens, secrets or passwords.

  From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Verbalized Sampling loads about 2k tokens when it runs, and up to ~11k if it reads all its reference files. Until then it costs about 164 tokens; SKILL.md has 811 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it

~164

When it runs · the whole SKILL.md, loaded when a task matches

~2k

With references · SKILL.md plus every file in references/, read only if the agent opens them

~11k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

• WarningTells the agent its actions are pre-authorized / not to stop for confirmationSKILL.md:63

  If you cannot answer Step 2 without asking the user, **ask first** before generating. Generic outputs caused by thin con

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.

SKILL.md

The full file from gnurio/nurijanian-skills at commit 43a0566, republished under its MIT licence (© gnurio). 811 words, ~1,974 tokens.

name

verbalized-sampling

description

Generate diverse outputs by prompting for a probability distribution instead of a single response. Implements Verbalized Sampling (VS) from Zhang et al. 2025 — a training-free technique that counteracts LLM mode collapse caused by typicality bias in alignment data. Use when the task needs genuine diversity: creative writing, brainstorming/ideation, synthetic data generation, persona/dialogue simulation, adversarial examples, open-ended QA with multiple valid answers, or any situation where "generate 5 ideas" keeps returning the same cluster. Do NOT use for: single correct answer tasks, factual lookup, strict format compliance.

[Task description with rich context]

Generate {k} responses. Return in JSON format with key "{output_key}" (list of dicts). Each dict:
• text: [output specification]
• probability: estimated probability (0.0–1.0) of this response given the input

{Distribution constraint}

Output ONLY the JSON object.

## High diversity (p < 0.05)
1. [text] (p=0.03)

## Moderate diversity (p 0.05–0.15)
2. [text] (p=0.08)

I need to generate diverse {output_type} for {use_case}.

Create a Verbalized Sampling prompt that:
1. Clearly describes the task with specific context about {use_case}
2. Requests k={number} outputs in JSON format
3. Requires each output to include "text" and "probability" fields
4. Specifies a distribution constraint appropriate for the diversity level needed:
   - 0.10–0.15 for moderate diversity
   - 0.05–0.10 for high diversity
   - 0.01–0.05 for maximum diversity
5. Ends with "Output ONLY the JSON"
6. Includes explicit problem-frame coverage if topic is likely overfit

© gnurio, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 5 other files (scripts, references) in skills/verbalized-sampling of gnurio/nurijanian-skills.

• SKILL.md
• references/critique-framework.md
• references/judges.md
• references/paper-insights.md
• references/templates.md
• scripts/format_vs_output.py

Open the folder on GitHubat commit 43a0566