Weavebench Cua Reproduce
AMAP-ML/LongHorizon-Harness
Reproduce CUA-Harness experiments on WeaveBench from a GitHub checkout.
Smoke test real Poracode provider chat threads and ACP sessions end to end.
$ npx skills add Porabuild/Poracode --skill provider-chat-smoke -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install Porabuild/Poracode provider-chat-smoke --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/Porabuild/Poracode.git skills-src && mkdir -p .claude/skills && cp -r skills-src/.agents/skills/provider-chat-smoke .claude/skills/provider-chat-smoke && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "provider-chat-smoke" agent skill from https://github.com/Porabuild/Poracode/tree/master/.agents/skills/provider-chat-smoke into .claude/skills/provider-chat-smoke/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "provider-chat-smoke", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/Porabuild/Poracode/tree/master/.agents/skills/provider-chat-smokeType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add Porabuild/Poracode --skill provider-chat-smoke -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install Porabuild/Poracode provider-chat-smoke --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/Porabuild/Poracode.git skills-src && mkdir -p .agents/skills && cp -r skills-src/.agents/skills/provider-chat-smoke .agents/skills/provider-chat-smoke && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "provider-chat-smoke" agent skill from https://github.com/Porabuild/Poracode/tree/master/.agents/skills/provider-chat-smoke into .agents/skills/provider-chat-smoke/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "provider-chat-smoke", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add Porabuild/Poracode --skill provider-chat-smoke -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install Porabuild/Poracode provider-chat-smoke --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/Porabuild/Poracode.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/.agents/skills/provider-chat-smoke .cursor/skills/provider-chat-smoke && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "provider-chat-smoke" agent skill from https://github.com/Porabuild/Poracode/tree/master/.agents/skills/provider-chat-smoke into .cursor/skills/provider-chat-smoke/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "provider-chat-smoke", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/Porabuild/Poracode.git --path .agents/skills/provider-chat-smoke--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add Porabuild/Poracode --skill provider-chat-smoke -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install Porabuild/Poracode provider-chat-smoke --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/Porabuild/Poracode.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/.agents/skills/provider-chat-smoke .gemini/skills/provider-chat-smoke && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "provider-chat-smoke" agent skill from https://github.com/Porabuild/Poracode/tree/master/.agents/skills/provider-chat-smoke into .gemini/skills/provider-chat-smoke/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "provider-chat-smoke", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install Porabuild/Poracode provider-chat-smokeInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add Porabuild/Poracode --skill provider-chat-smoke -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/Porabuild/Poracode.git skills-src && mkdir -p .github/skills && cp -r skills-src/.agents/skills/provider-chat-smoke .github/skills/provider-chat-smoke && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "provider-chat-smoke" agent skill from https://github.com/Porabuild/Poracode/tree/master/.agents/skills/provider-chat-smoke into .github/skills/provider-chat-smoke/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "provider-chat-smoke", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add Porabuild/Poracode --skill provider-chat-smoke -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install Porabuild/Poracode provider-chat-smoke --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/Porabuild/Poracode.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/.agents/skills/provider-chat-smoke .opencode/skills/provider-chat-smoke && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "provider-chat-smoke" agent skill from https://github.com/Porabuild/Poracode/tree/master/.agents/skills/provider-chat-smoke into .opencode/skills/provider-chat-smoke/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "provider-chat-smoke", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
provider-chat-smokeSmoke test real Poracode provider chat threads and ACP sessions end to end.
Provider Chat Smoke is an agent skill from Porabuild/Poracode. Smoke test real Poracode provider chat threads and ACP sessions end to end. Use when validating Qwen Code, Kimi Code, or another structured/ACP provider; testing chat turn handling, steer, Stop, question or permission tools, live model changes, session resume, ACP handshake/capabilities, or provider-chat regressions.
Its SKILL.md is about 4.2k tokens, which your agent loads only when the skill is triggered. The skill folder holds 2 other files (for example `agents/openai.yaml`).
It sits in Testing & QA, covering QA and bug reports. It works with Kimi and Qwen. The repository describes itself as: One window for all your AI coding agents. Run Claude, Codex, OpenCode, Gemini, Antigravity, Cursor, and Copilot side-by-side. Terminal and chat, any layout. The licence is Apache-2.0.
4 steps, taken from the first numbered list in SKILL.md.
Read from SKILL.md and the folder at commit e56fcc8. It shows what the files ask for, not the result of running them.
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
No scripts in the folder and no shell commands in SKILL.md.
From the folder's file list and the shell code blocks in SKILL.md.
No URLs in SKILL.md.
From URLs in SKILL.md, links to its own repository left out.
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Provider Chat Smoke loads about 4.2k tokens when it runs. Until then it costs about 85 tokens; SKILL.md has 1,422 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.
The full file from Porabuild/Poracode at commit e56fcc8, republished under its Apache-2.0 licence (© Porabuild). 1,422 words, ~4,168 tokens.
.claude/skills/provider-chat-smoke/SKILL.md (or your agent's skills folder). This skill also uses 1 other file; get the full folder from GitHub.Use this skill with interactive-testing. Run the ordinary isolated-app workflow from that skill first; use real provider credentials only for the safe live gates below. Keep the scope to a disposable project, never approve a write, and preserve the smoke profile for inspection.
QWEN_SMOKE_OK and KIMI_SMOKE_OK; never use repository-changing prompts.Run each applicable row for every requested provider. Retry a timing-sensitive row with a longer harmless response if the provider finishes before the control can be used.
| Gate | Drive through the real UI | Required evidence |
|---|---|---|
| Launch and first turn | Choose the provider/model, submit a marker prompt, and wait for idle. | User row, matching assistant marker, no error state. |
| Image attachment turn | Paste/attach an image in the composer of the first turn (marker prompt referencing it), wait for idle. | Image chip on the user row, the assistant actually reads/acknowledges the image content (providers with readsImageAttachmentsFromHost: false read the localized file via their own tools), and no media-rejection error state. |
| Commands, skills, plugins | Type / in the composer: provider-native slash commands, skills, and plugin contributions must appear. | Provider-declared commands/skills render (from the slashCommands capability or a provider-owned catalog via reportsSkillCatalog); picking one submits its invocation. If the provider exposes none, report that with the capability snapshot — do not accept a bare generic list as PASS. |
| Sub-agent rendering | Trigger a sub-agent run (a prompt that spawns subtasks, e.g. parallel research), wait for completion. | subagent_*/Task-style tool calls render as sub-agent rows in the composer and the thread panel (never as plain tool rows), and child activity stays inside the agent row/overlay instead of flooding the main timeline. |
| Follow-up | Send a second marker in the same thread. | Ordered second user/assistant pair. |
| Structured question | Ask the agent to invoke its question tool with two choices; submit a benign answer. | Question dock, selected answer, turn resumes and completes. Submit must be disabled until every question is answered — an empty submit must never reach the supervisor. |
| Permission | Ask for exactly pwd (or another read-only command), wait for the approval dock, then choose Reject/Deny. | Approval details, declined tool/command item, agent confirms it did not run (for filesystem-verifiable actions, check the file was not created). |
| Steer | Start a long harmless response; while status is working, send a replacement instruction with a marker. | Working state, pending-steer strip or equivalent, original turn cancels, replacement marker completes. |
| Stop | Start a long harmless response, press Stop response promptly, and wait for terminal state. | Stop control accepted, no endlessly-working thread, cancellation/idle/error outcome recorded. |
| Mid-thread model change | Between completed turns, change to a different model offered by the same provider, then send a marker prompt. | Picker shows the new model, follow-up completes, persisted thread config has the new model. |
| Provider switch | Build source history on another provider (include an image turn), then use the thread header's Continue in another provider to switch in place and send a marker. | Thread continues on the target provider, prior context transfers, no route/sessionInUse error. The in-thread model picker is provider-locked on active threads — switching goes through the header control. |
| Reopen/resume | Leave the thread (close the pane), reopen it, and send one more harmless prompt. | Existing transcript is not duplicated, session reference remains usable, reply completes. A thread that sits "working" for minutes with no item activity and no error is a hang — FAIL, not slowness. |
Do not substitute a successful completion for Stop: a response that ends before the Stop click is not tested. Do not silently claim a provider supports questions, permissions, or model changes when its capabilities did not expose them; report those as not applicable with the advertised capability snapshot.
For a real ACP provider, prove the full path rather than only the rendered reply:
createStructuredSession() succeeds and that session/new or session/load yields a stable session reference. For Kimi, verify its discovered providerSessionId is persisted after its delayed session-file discovery.session/update messages produce the expected canonical runtime items: user message, reasoning, assistant message, tool call, and tool result/command result when applicable.request.resolved outcome: answered for a question, declined for a rejected approval, or cancelled for an abandoned request.Capture session identity and configs without copying credentials, bearer tokens, or raw sensitive environment values into artifacts.
Cover every branch below with focused tests/mocks when the live provider does not safely expose it. Add or keep a production scenario mapping for any new provider surface.
| Area | Branches to cover |
|---|---|
| Handshake | spawn failure, initialize/protocol mismatch, capability probe timeout, authenticated/missing auth, session/new, and session/load. |
| Session lifecycle | GUI resume versus terminal resume gating, known/discovered session refs, invalid/expired session recovery, process exit/connection loss, dispose, no duplicate history on replay, and failed restart settles the submitted turn (a restart/resume failure — e.g. a sessionInUse-style lock error — must append a visible error item and errored thread state; it must never strand the thread "working" with the send lost). |
| WSL execution | For providers executing through WSL (windowsProjectExecution: "wsl"): the Linux-side host must not survive teardown — killing the Windows wrapper is not enough, so dispose kills the surviving Linux process (host-cookie environ sweep via the bridge) and no orphan host is left holding provider-side session locks after thread stop or app stop. |
| Provider routing | When the provider protocol has internal model/provider routing (e.g. Muse session/start.providerId), launch params must carry the catalog-resolved route rather than the server default — defaults can differ in capability (one Muse route rejects media in retained history, the catalog route accepts it). |
| Updates | assistant text, reasoning, tool call/update/result, plan/file-change updates, malformed/noise filtering, provider-specific transforms, and empty-response error rewriting. |
| Config sync | model, mode, effort/thought-level changes through config options; explicit response versus later notification; unsupported/unstable-model fallback; rejected/timeout config updates. |
| Prompts | normal completion, RPC error, agent-visible error, cancellation before prompt acceptance, cancellation after activity, pending-steer replacement semantics, Stop watchdog, stale interrupt immunity after restart, and image-attachment delivery per readsImageAttachmentsFromHost (inline image content versus @path mention after execution-location rewrite). |
| Requests | permission accept/deny/cancel, question options/custom answer/skip (the question form must not submit unanswered questions — the supervisor rejects empty answers), createElicitation, completeElicitation, request resolution after teardown, and synthetic auto-approval only where the configured policy permits it. |
| Client services | client-hosted terminal create/write/output/wait/release, read/write resource path validation, and MCP config/launch gating. Test writes only with the fixture repository. |
When a live turn fails, classify the error before writing the verdict — some rejections come from the provider's own model routing, not from Poracode:
provider-private history is incompatible with the active route: retained media history is unsupported … — the provider's model route refused media in
the conversation history. Poracode delivered an image the route cannot take;
fix delivery (path mention + readsImageAttachmentsFromHost: false), not the
error handling.session … is already in use (sessionInUse) — another live host holds the
provider-side session lock, typically an orphaned WSL host that teardown
failed to kill. Poracode must surface this as a turn error, never hang.Report a verdict per provider and per gate: PASS, FAIL, SKIPPED, or NOT APPLICABLE. Include connection/capability evidence, session-reference continuity, error count with the first three errors, and screenshot/report paths. Distinguish real provider evidence from mocked protocol coverage.
For WSL-executing providers, teardown hygiene is part of the verdict: after stopping a thread and after stopping the app, check the distro for surviving provider host processes (ps aux | grep <binary>) — a survivor holds provider-side locks and breaks the next resume.
Reset the isolated profile and stop only the process launched for this run. Leave the smoke directory intact unless cleanup is requested.
© Porabuild, Apache-2.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
SKILL.md and 1 other file in .agents/skills/provider-chat-smoke of Porabuild/Poracode.
Open the folder on GitHubat commit e56fcc8
Provider Chat Smoke next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Provider Chat Smoke this skillPorabuild/Poracode | 115 | — | ~4.2k | Automated safety check: Pass | Apache-2.0 | |
| Weavebench Cua ReproduceAMAP-ML/LongHorizon-Harness | 1.7k | — | ~1.6k | Automated safety check: Pass | MIT | |
| tmux Real User TestingQwenLM/qwen-code | 28k | — | ~2.3k | Automated safety check: Pass | Apache-2.0 | |
| Run Shuntpleaseai/shunt | 203 | — | ~2.6k | Automated safety check: Pass | Apache-2.0 | |
| Qwen Mtp Gguf ReleaseR6410418/Jackrong-llm-finetuning-guide | 1.7k | — | ~474 | Automated safety check: Pass | Apache-2.0 | |
| Qwen Mtp GgufR6410418/Jackrong-llm-finetuning-guide | 1.7k | — | ~1.7k | Automated safety check: Pass | MIT |
AMAP-ML/LongHorizon-Harness
Reproduce CUA-Harness experiments on WeaveBench from a GitHub checkout.
QwenLM/qwen-code
Drives Qwen Code in a real tmux session the way a user would and saves a readable step-by-step transcript of each screen for maintainers to review.
pleaseai/shunt
Build, launch, and drive shunt — the Claude Code LLM gateway (a Rust/axum Anthropic-Messages proxy).
R6410418/Jackrong-llm-finetuning-guide
Repository-level wrapper for the canonical Qwen MTP or nextn GGUF release workflow.
R6410418/Jackrong-llm-finetuning-guide
Complete agent-ready workflow for Qwen-family MTP or nextn GGUF conversion and release.
luoyuctl/agenttrace
Audit local AI coding-agent sessions with agenttrace. An agent skill from luoyuctl/agenttrace.
Porabuild/Poracode
Run repeatable integration and smoke testing against the real Poracode Electron app through Chrome DevTools Protocol.
Porabuild/Poracode
Write consistent, hand-written-looking changelog entries for a Poracode release.
Porabuild/Poracode
Create a new reusable agent skill managed by Poracode. An agent skill from Porabuild/Poracode.
Porabuild/Poracode
Inspect and drive Poracode itself — the Terminal panel, other threads, projects, workspaces, git, pull requests, and schedules — through the poracode MCP.
Porabuild/Poracode
Use the user's real Chrome tabs and signed-in sessions for visible browser workflows.
Porabuild/Poracode
Inspect and operate native Windows, macOS, or Linux applications through Poracode's desktop-control tools.
Categories
Smoke test real Poracode provider chat threads and ACP sessions end to end. Provider Chat Smoke is an agent skill from Porabuild/Poracode. Smoke test real Poracode provider chat threads and ACP sessions end to end.
Provider Chat Smoke fits situations like: validating Qwen Code; another structured/ACP provider; testing chat turn handling; permission tools.
Run `npx skills add Porabuild/Poracode --skill provider-chat-smoke -a claude-code`. Or copy the skill folder (.agents/skills/provider-chat-smoke in Porabuild/Poracode) into .claude/skills/provider-chat-smoke in your project. Claude Code loads it when a task matches its description.
Run `npx skills add Porabuild/Poracode --skill provider-chat-smoke -a codex`. Or copy the skill folder (.agents/skills/provider-chat-smoke in Porabuild/Poracode) into .agents/skills/provider-chat-smoke in your project. Codex loads it when a task matches its description.
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add Porabuild/Poracode --skill provider-chat-smoke -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/provider-chat-smoke, .gemini/skills/provider-chat-smoke, .github/skills/provider-chat-smoke and .opencode/skills/provider-chat-smoke in your project.
SKILL.md names no scripts, command-line tools or credentials: Provider Chat Smoke is instructions for the agent only.
SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.
Provider Chat Smoke is published under the Apache-2.0 licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.
About 4.2k tokens (SKILL.md is roughly 17k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.
Skills that share tags, products or a category with Provider Chat Smoke: Weavebench Cua Reproduce (AMAP-ML/LongHorizon-Harness, 1.7k stars), tmux Real User Testing (QwenLM/qwen-code, 28k stars), Run Shunt (pleaseai/shunt, 203 stars) and Qwen Mtp Gguf Release (R6410418/Jackrong-llm-finetuning-guide, 1.7k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
Porabuild (a GitHub organization) maintains it in Porabuild/Poracode, which has 115 GitHub stars. The repository holds 21 skills in this directory. The repository was last updated on October 10, 2026.
Source: Porabuild/Poracode on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.