OpenHarness End-to-End Evals
HKUDS/OpenHarness
Validates OpenHarness features by running real multi-turn agent loops with live LLM calls against an unfamiliar codebase, checking actual tool execution.
Upgrade shared Modal runtime dependencies in kernelbot and verify them end to end.
Install Modal Runtime Deploy E2E
$ npx skills add gpu-mode/kernelbot --skill modal-runtime-deploy-e2e -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install gpu-mode/kernelbot modal-runtime-deploy-e2e --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/gpu-mode/kernelbot.git skills-src && mkdir -p .claude/skills && cp -r skills-src/.codex/skills/modal-runtime-deploy-e2e .claude/skills/modal-runtime-deploy-e2e && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "modal-runtime-deploy-e2e" agent skill from https://github.com/gpu-mode/kernelbot/tree/main/.codex/skills/modal-runtime-deploy-e2e into .claude/skills/modal-runtime-deploy-e2e/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "modal-runtime-deploy-e2e", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/gpu-mode/kernelbot/tree/main/.codex/skills/modal-runtime-deploy-e2eType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add gpu-mode/kernelbot --skill modal-runtime-deploy-e2e -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install gpu-mode/kernelbot modal-runtime-deploy-e2e --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/gpu-mode/kernelbot.git skills-src && mkdir -p .agents/skills && cp -r skills-src/.codex/skills/modal-runtime-deploy-e2e .agents/skills/modal-runtime-deploy-e2e && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "modal-runtime-deploy-e2e" agent skill from https://github.com/gpu-mode/kernelbot/tree/main/.codex/skills/modal-runtime-deploy-e2e into .agents/skills/modal-runtime-deploy-e2e/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "modal-runtime-deploy-e2e", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add gpu-mode/kernelbot --skill modal-runtime-deploy-e2e -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install gpu-mode/kernelbot modal-runtime-deploy-e2e --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/gpu-mode/kernelbot.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/.codex/skills/modal-runtime-deploy-e2e .cursor/skills/modal-runtime-deploy-e2e && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "modal-runtime-deploy-e2e" agent skill from https://github.com/gpu-mode/kernelbot/tree/main/.codex/skills/modal-runtime-deploy-e2e into .cursor/skills/modal-runtime-deploy-e2e/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "modal-runtime-deploy-e2e", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/gpu-mode/kernelbot.git --path .codex/skills/modal-runtime-deploy-e2e--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add gpu-mode/kernelbot --skill modal-runtime-deploy-e2e -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install gpu-mode/kernelbot modal-runtime-deploy-e2e --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/gpu-mode/kernelbot.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/.codex/skills/modal-runtime-deploy-e2e .gemini/skills/modal-runtime-deploy-e2e && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "modal-runtime-deploy-e2e" agent skill from https://github.com/gpu-mode/kernelbot/tree/main/.codex/skills/modal-runtime-deploy-e2e into .gemini/skills/modal-runtime-deploy-e2e/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "modal-runtime-deploy-e2e", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install gpu-mode/kernelbot modal-runtime-deploy-e2eInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add gpu-mode/kernelbot --skill modal-runtime-deploy-e2e -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/gpu-mode/kernelbot.git skills-src && mkdir -p .github/skills && cp -r skills-src/.codex/skills/modal-runtime-deploy-e2e .github/skills/modal-runtime-deploy-e2e && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "modal-runtime-deploy-e2e" agent skill from https://github.com/gpu-mode/kernelbot/tree/main/.codex/skills/modal-runtime-deploy-e2e into .github/skills/modal-runtime-deploy-e2e/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "modal-runtime-deploy-e2e", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add gpu-mode/kernelbot --skill modal-runtime-deploy-e2e -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install gpu-mode/kernelbot modal-runtime-deploy-e2e --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/gpu-mode/kernelbot.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/.codex/skills/modal-runtime-deploy-e2e .opencode/skills/modal-runtime-deploy-e2e && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "modal-runtime-deploy-e2e" agent skill from https://github.com/gpu-mode/kernelbot/tree/main/.codex/skills/modal-runtime-deploy-e2e into .opencode/skills/modal-runtime-deploy-e2e/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "modal-runtime-deploy-e2e", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
Facts
- Skill name
modal-runtime-deploy-e2e- GitHub stars
- 114
- Token cost
- ~748 tokens
- SKILL.md length
- 196 words
- Files
- 1
- Skills in repo
- 1
- Repo updated
- First seen
- Licence
- Custom licence
At a glance
Upgrade shared Modal runtime dependencies in kernelbot and verify them end to end.
- Works in 8 steps: Make the smallest dependency change in… → If changing torch/CUDA, inspect all… → Deploy to Modal pytest first. → …
- Changing torch/CUDA
- SKILL.md covers Scope, Workflow, Closed Leaderboards and Real E2E Submit, plus 1 more section
- Calls cargo, uv and modal; needs MODAL_TOKEN_SECRET and POPCORN_ADMIN_TOKEN
What it does
Modal Runtime Deploy E2E is an agent skill from gpu-mode/kernelbot. Upgrade shared Modal runtime dependencies in kernelbot and verify them end to end. Use when changing torch/CUDA or other shared Modal image dependencies, deploying the Modal app, and validating with both Modal integration tests and real popcorn leaderboard submissions.
Its SKILL.md is about 750 tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.
It sits in Testing & QA, covering End-to-end testing and Integration testing. It works with CUDA. The repository describes itself as: Write a fast kernel and see how you compare against the best humans and AI on gpumode.com.
When your agent uses it
- Changing torch/CUDA
- Other shared Modal image dependencies
- Deploying the Modal app
- Validating with both Modal integration tests and real popcorn leaderboard submissions
Example prompts
- “/modal-runtime-deploy-e2e”
Requirements
- Python 3
- A credential in MODAL_TOKEN_SECRET
- A credential in POPCORN_ADMIN_TOKEN
Workflow steps
8 steps, taken from the first numbered list in SKILL.md.
- Make the smallest dependency change in src/runners/modal_runner.py.
- If changing torch/CUDA, inspect all later .uv_pip_install(...) blocks for conflicting CUDA/NCCL packages.
- Deploy to Modal pytest first.
- Run the narrow Modal integration test
- If that passes, deploy to Modal main
- Run a real popcorn submission in test mode against the target leaderboard.
- Confirm the returned report shows the expected Torch: version.
- Only then run --mode leaderboard if the user asked for a ranked submission.
What it can do on your machine
Read from SKILL.md and the folder at commit ec46696. It shows what the files ask for, not the result of running them.
Tool permissions
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Runs code
Shell commands in SKILL.md call:
cargouvmodal
From the folder's file list and the shell code blocks in SKILL.md.
Network
No URLs in SKILL.md. Its commands use uv, which can reach the network depending on how they are called.
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names these keys or tokens, usually read from environment variables:
MODAL_TOKEN_SECRETPOPCORN_ADMIN_TOKEN
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Context cost
Modal Runtime Deploy E2E loads about 748 tokens when it runs. Until then it costs about 74 tokens; SKILL.md has 196 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
Safety
Auto-check passed
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.
SKILL.md
Its licence (Custom licence) doesn't allow us to republish the file, so here is its outline and opening line. It has 196 words (~748 tokens).
“Use this when changing shared Modal dependencies in kernelbot, especially torch/CUDA, and when you need to prove the live leaderboard is actually using the new runtime.”
- name
- modal-runtime-deploy-e2e
Files
Just SKILL.md in .codex/skills/modal-runtime-deploy-e2e of gpu-mode/kernelbot.
Open the folder on GitHubat commit ec46696
Compare with similar skills
Modal Runtime Deploy E2E next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Modal Runtime Deploy E2E this skillgpu-mode/kernelbot | 114 | — | ~748 | Automated safety check: Pass | Custom licence | |
| OpenHarness End-to-End EvalsHKUDS/OpenHarness | 16k | 1 repos | ~2.1k | Automated safety check: Notes | MIT | |
| E2E Testinglangflow-ai/langflow | 156k | — | ~3.3k | Automated safety check: Pass | MIT | |
| MongoDB Source Connector E2E Harnessairbytehq/airbyte | 22k | — | ~1.9k | Automated safety check: Pass | Custom licence | |
| Java SDK E2E Test with Replay Snapshotgithub/copilot-sdk | 11k | — | ~1.8k | Automated safety check: Pass | MIT | |
| Go Redis Client Test Runnerredis/go-redis | 22k | — | ~786 | Automated safety check: Pass | BSD-2-Clause |
Similar skills
E2E Testing
langflow-ai/langflow
Write and review Playwright E2E tests for Langflow. An agent skill from langflow-ai/langflow.
- Official
MongoDB Source Connector E2E Harness
airbytehq/airbyte
Starts a throwaway MongoDB 7.0 replica set and runs the Airbyte spec, check, discover and read commands against source-mongodb-v2 images for local end-to-end testing.
- Official
Java SDK E2E Test with Replay Snapshot
github/copilot-sdk
Creates a Java SDK end-to-end test for the Copilot SDK that runs against a recorded YAML snapshot through a replay proxy, so CI needs no real authentication.
- Official
Go Redis Client Test Runner
redis/go-redis
Explains how to run go-redis tests: the Docker Compose stack, make targets, focusing a single Ginkgo spec, the e2e suite and the version environment variables.
- Official
Airbyte Postgres Source E2E Tests
airbytehq/airbyte
Starts a local PostgreSQL 16 container, loads SQL fixtures and runs the Airbyte spec, check, discover and read commands against a chosen source-postgres image.
Related
Works with
Categories
Questions about Modal Runtime Deploy E2E
What does Modal Runtime Deploy E2E do?
Upgrade shared Modal runtime dependencies in kernelbot and verify them end to end. Modal Runtime Deploy E2E is an agent skill from gpu-mode/kernelbot. Upgrade shared Modal runtime dependencies in kernelbot and verify them end to end.
When should I use Modal Runtime Deploy E2E?
Modal Runtime Deploy E2E fits situations like: changing torch/CUDA; other shared Modal image dependencies; deploying the Modal app; validating with both Modal integration tests and real popcorn leaderboard submissions.
How do I install Modal Runtime Deploy E2E in Claude Code?
Run `npx skills add gpu-mode/kernelbot --skill modal-runtime-deploy-e2e -a claude-code`. Or copy the skill folder (.codex/skills/modal-runtime-deploy-e2e in gpu-mode/kernelbot) into .claude/skills/modal-runtime-deploy-e2e in your project. Claude Code loads it when a task matches its description.
How do I install Modal Runtime Deploy E2E in Codex?
Run `npx skills add gpu-mode/kernelbot --skill modal-runtime-deploy-e2e -a codex`. Or copy the skill folder (.codex/skills/modal-runtime-deploy-e2e in gpu-mode/kernelbot) into .agents/skills/modal-runtime-deploy-e2e in your project. Codex loads it when a task matches its description.
Can I use Modal Runtime Deploy E2E in Cursor, Gemini CLI or GitHub Copilot?
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add gpu-mode/kernelbot --skill modal-runtime-deploy-e2e -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/modal-runtime-deploy-e2e, .gemini/skills/modal-runtime-deploy-e2e, .github/skills/modal-runtime-deploy-e2e and .opencode/skills/modal-runtime-deploy-e2e in your project.
What does Modal Runtime Deploy E2E need to run?
Going by SKILL.md and its folder, Modal Runtime Deploy E2E needs the command-line tools its instructions call (cargo, uv and modal) and credentials named MODAL_TOKEN_SECRET and POPCORN_ADMIN_TOKEN. Our summary lists: Python 3; A credential in MODAL_TOKEN_SECRET; A credential in POPCORN_ADMIN_TOKEN.
Does Modal Runtime Deploy E2E access the network?
SKILL.md contains no URLs. Its commands use uv, which can reach the network depending on how they are called. This is read from the text; nothing was executed.
Is Modal Runtime Deploy E2E safe to install?
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.
What licence does Modal Runtime Deploy E2E use?
Modal Runtime Deploy E2E has a licence file (the repository's licence) that doesn't match a standard licence. Read it on GitHub before reusing the skill.
How many tokens does Modal Runtime Deploy E2E use?
About 748 tokens (SKILL.md is roughly 3k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.
What are the alternatives to Modal Runtime Deploy E2E?
Skills that share tags, products or a category with Modal Runtime Deploy E2E: OpenHarness End-to-End Evals (HKUDS/OpenHarness, 16k stars), E2E Testing (langflow-ai/langflow, 156k stars), MongoDB Source Connector E2E Harness (airbytehq/airbyte, 22k stars) and Java SDK E2E Test with Replay Snapshot (github/copilot-sdk, 11k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
Who maintains Modal Runtime Deploy E2E?
gpu-mode (a GitHub organization) maintains it in gpu-mode/kernelbot, which has 114 GitHub stars. The repository was last updated on September 18, 2026.
Source: gpu-mode/kernelbot on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.