Generate Verifiers Env

What it does

When your agent uses it

• Someone asks to scaffold a Verifiers env
• Port to Verifiers
• Build an in-process toolkit
• Set up a vf.ToolEnv with a Rubric

Example prompts

• “make a verifiers env for X”
• “wrap my game in verifiers”
• “set up a vf.ToolEnv”
• “/generate-verifiers-env”

Requirements

• Python 3
• Docker

Workflow steps

4 steps, taken from the step headings in SKILL.md.

1. 1The toolkit class
2. 2Standalone tool functions for vf.ToolEnv
3. 3The rubric
4. 4Rollout — rollout.py

What it can do on your machine

Read from SKILL.md and the folder at commit b0f4c2f. It shows what the files ask for, not the result of running them.

• Tool permissions

  Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

  From allowed-tools in the SKILL.md frontmatter.

• Runs code

  Shell commands in SKILL.md call:

  • uv

  From the folder's file list and the shell code blocks in SKILL.md.

• Network

  Links to these hosts (documentation or services it may open):

  • github.com
  • docs.primeintellect.ai
  • pypi.org

  From URLs in SKILL.md, links to its own repository left out.

• Credentials

  Names no API keys, tokens, secrets or passwords.

  From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Generate Verifiers Env loads about 2.3k tokens when it runs, and up to ~3.6k if it reads all its reference files. Until then it costs about 207 tokens; SKILL.md has 629 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it

~207

When it runs · the whole SKILL.md, loaded when a task matches

~2.3k

With references · SKILL.md plus every file in references/, read only if the agent opens them

~3.6k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from adithya-s-k/FineEnvs at commit b0f4c2f, republished under its Apache-2.0 licence (© adithya-s-k). 629 words, ~2,322 tokens.

name

generate-verifiers-env

description

Builds a Verifiers (PrimeIntellect) variant of an RL environment. Use whenever someone asks to scaffold a Verifiers env, port to Verifiers, build an in-process toolkit, set up a `vf.ToolEnv` with a Rubric, or wire up a TRL `GRPOTrainer` rollout. Verifiers is the right framework when the user wants in-process tools (no HTTP server), structured tool calling driven by plain Python functions, composable reward rubrics with multiple grader functions, fast iteration with no Docker, or the cleanest path from prototype to TRL training. Output is a runnable `<env_dir>/verifiers/` folder with `env.py` (toolkit + standalone tool functions + `create_verifiers_env`), `rollout.py`, and `pyproject.toml`. Use for prompts like "make a verifiers env for X", "wrap my game in verifiers", or "set up a vf.ToolEnv".

class WordleToolkit:
    def __init__(self): ...
    def initialize(self): ...     # lazy E2B / state init
    def cleanup(self): ...        # kill sandbox
    def reset(self): ...          # new episode
    def guess(self, word: str) -> str:
        """Submit a 5-letter word guess. Returns colored feedback."""
        ...

def create_verifiers_env():
    import verifiers as vf
    from datasets import Dataset
    dataset = Dataset.from_list([{"question": t["task"], "answer": t["expected_output"]} for t in TASKS])
    async def correctness(completion, answer, **kwargs) -> float:
        # read from the completion trajectory; return 0.0–1.0
        ...
    rubric = vf.Rubric(funcs=[correctness])
    return vf.ToolEnv(tools=TOOL_FUNCTIONS, max_turns=8, dataset=dataset, rubric=rubric, system_prompt="...")

<env_dir>/verifiers/
├── pyproject.toml      # verifiers + e2b-* + datasets + python-dotenv + openai
├── __init__.py
├── env.py              # Toolkit class + standalone tool fns + create_verifiers_env()
├── rollout.py          # Drives the toolkit manually with the openai client
└── README.md

_shared: Optional[WordleToolkit] = None
def _kit():
    global _shared
    if _shared is None:
        _shared = WordleToolkit()
    return _shared

def guess(word: str) -> str:
    """Submit a 5-letter word guess."""
    return _kit().guess(word)

TOOL_FUNCTIONS = [guess]

async def correctness(completion, answer, **kwargs) -> float:
    if not completion: return 0.0
    last = completion[-1].get("content", "") if isinstance(completion[-1], dict) else str(completion[-1])
    return 1.0 if answer.strip() in last.strip() else 0.0

async def correctness(completion, answer, **kwargs) -> float:
    seen_success = any("terminated: success" in str(m) for m in completion)
    seen_expected = any(answer in str(m) for m in completion)
    return 1.0 if (seen_success and seen_expected) else (0.5 if seen_success else 0.0)

def func_to_openai_tool(fn):
    sig = inspect.signature(fn)
    hints = get_type_hints(fn)
    doc = (fn.__doc__ or "").strip().split("\n\n")[0]
    properties, required = {}, []
    for name, p in sig.parameters.items():
        ann = hints.get(name, str)
        origin = get_origin(ann)
        if origin in (list, "list"):
            inner = get_args(ann)
            properties[name] = {"type": "array", "items": {"type": "integer" if (inner and inner[0] is int) else "string"}}
        elif ann is int:    properties[name] = {"type": "integer"}
        elif ann is float:  properties[name] = {"type": "number"}
        elif ann is bool:   properties[name] = {"type": "boolean"}
        else:               properties[name] = {"type": "string"}
        if p.default is inspect.Parameter.empty:
            required.append(name)
    return {"type": "function", "function": {
        "name": fn.__name__, "description": doc,
        "parameters": {"type": "object", "properties": properties, "required": required},
    }}

text, b64 = kit._ctrl.screenshot()      # if you exposed _ctrl
messages.append({"role": "user", "content": [
    {"type": "text", "text": "Latest screenshot:"},
    {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{b64}"}},
]})

© adithya-s-k, Apache-2.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 1 other file (references) in .claude/skills/generate-verifiers-env of adithya-s-k/FineEnvs.

• SKILL.md
• references/architecture.md

Open the folder on GitHubat commit b0f4c2f

Used in 1 other repository

We found 1 copy of this SKILL.md (exact, near-identical or edited) in other folders, from 1 other GitHub owner. This page covers the copy in adithya-s-k/FineEnvs, which our catalogue first saw on October 7, 2026.

• NeverSight/learn-skills.devdata/skills-md/adithya-s-k/rl_envs_101/generate-verifiers-env