Agent skill

Gui Automation

MITAuto-check passedProductivity & Automation

Install Gui Automation

skills CLI

$ npx skills add am-will/gooey-pi --skill gui-automation -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI

$ gh skill install am-will/gooey-pi gui-automation --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy

$ git clone --depth 1 https://github.com/am-will/gooey-pi.git skills-src && mkdir -p .claude/skills && cp -r skills-src/assets/skills/gui-automation .claude/skills/gui-automation && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Paste into Claude Code

Install the "gui-automation" agent skill from https://github.com/am-will/gooey-pi/tree/main/assets/skills/gui-automation into .claude/skills/gui-automation/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "gui-automation", then confirm the skill loads.

Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

$skill-installer

$skill-installer install https://github.com/am-will/gooey-pi/tree/main/assets/skills/gui-automation

Type this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.

skills CLI

$ npx skills add am-will/gooey-pi --skill gui-automation -a codex

Project install goes to .agents/skills/; add -g for ~/.codex/skills/.

GitHub CLI

$ gh skill install am-will/gooey-pi gui-automation --agent codex

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/am-will/gooey-pi.git skills-src && mkdir -p .agents/skills && cp -r skills-src/assets/skills/gui-automation .agents/skills/gui-automation && rm -rf skills-src

Use ~/.agents/skills/ instead of .agents/skills for a personal install.

Codex skills documentation · loads skills from .agents/skills/

Paste into Codex

Install the "gui-automation" agent skill from https://github.com/am-will/gooey-pi/tree/main/assets/skills/gui-automation into .agents/skills/gui-automation/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "gui-automation", then confirm the skill loads.

Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

skills CLI

$ npx skills add am-will/gooey-pi --skill gui-automation -a cursor

Project install goes to .agents/skills/; add -g for ~/.cursor/skills/.

GitHub CLI

$ gh skill install am-will/gooey-pi gui-automation --agent cursor

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/am-will/gooey-pi.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/assets/skills/gui-automation .cursor/skills/gui-automation && rm -rf skills-src

Use ~/.cursor/skills/ instead of .cursor/skills for a personal install.

Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/

Paste into Cursor

Install the "gui-automation" agent skill from https://github.com/am-will/gooey-pi/tree/main/assets/skills/gui-automation into .cursor/skills/gui-automation/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "gui-automation", then confirm the skill loads.

Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

gemini skills install

$ gemini skills install https://github.com/am-will/gooey-pi.git --path assets/skills/gui-automation

--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.

skills CLI

$ npx skills add am-will/gooey-pi --skill gui-automation -a gemini-cli

Project install goes to .agents/skills/; add -g for ~/.gemini/skills/.

GitHub CLI

$ gh skill install am-will/gooey-pi gui-automation --agent gemini-cli

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/am-will/gooey-pi.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/assets/skills/gui-automation .gemini/skills/gui-automation && rm -rf skills-src

Use ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.

Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/

Paste into Gemini CLI

Install the "gui-automation" agent skill from https://github.com/am-will/gooey-pi/tree/main/assets/skills/gui-automation into .gemini/skills/gui-automation/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "gui-automation", then confirm the skill loads.

Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

GitHub CLI

$ gh skill install am-will/gooey-pi gui-automation

Installs for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).

skills CLI

$ npx skills add am-will/gooey-pi --skill gui-automation -a github-copilot

Project install goes to .agents/skills/; add -g for ~/.copilot/skills/.

Manual copy

$ git clone --depth 1 https://github.com/am-will/gooey-pi.git skills-src && mkdir -p .github/skills && cp -r skills-src/assets/skills/gui-automation .github/skills/gui-automation && rm -rf skills-src

Use ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.

GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/

Paste into GitHub Copilot

Install the "gui-automation" agent skill from https://github.com/am-will/gooey-pi/tree/main/assets/skills/gui-automation into .github/skills/gui-automation/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "gui-automation", then confirm the skill loads.

GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

skills CLI

$ npx skills add am-will/gooey-pi --skill gui-automation -a opencode

OpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.

GitHub CLI

$ gh skill install am-will/gooey-pi gui-automation --agent opencode

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/am-will/gooey-pi.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/assets/skills/gui-automation .opencode/skills/gui-automation && rm -rf skills-src

Use ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.

OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/

Paste into OpenCode

Install the "gui-automation" agent skill from https://github.com/am-will/gooey-pi/tree/main/assets/skills/gui-automation into .opencode/skills/gui-automation/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "gui-automation", then confirm the skill loads.

OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

Facts

Skill name: gui-automation
GitHub stars: 941
Token cost: ~1.4k tokens
SKILL.md length: 354 words
Files: 1
Skills in repo: 8
Repo updated: Sep 9, 2026
First seen: Oct 7, 2026
Licence: MIT

At a glance

You need to visually interact with a GUI — test buttons
SKILL.md covers Setup, Workflow, Scenarios and Trajectory, plus 2 more sections
Needs ANTHROPIC_API_KEY
Verify visual layouts

What it does

Gui Automation is an agent skill from am-will/gooey-pi. Use when you need to visually interact with a GUI — test buttons, fill forms, verify visual layouts, fuzz web pages, automate user flows, take screenshots, or perform end-to-end QA on any application. Works on cloud VMs, Docker containers, local machines, and sandboxes. Install: pip install cua.

Its SKILL.md is about 1.4k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.

It sits in Productivity & Automation, covering Desktop control, UX design and Forms and invoices. It works with Docker. The repository describes itself as: Desktop workspace for Pi, OMP, and Prime Agent. The licence is MIT.

When your agent uses it

You need to visually interact with a GUI — test buttons
Verify visual layouts
Automate user flows
Take screenshots

Example prompts

“/gui-automation”

Requirements

Python 3
Docker
A credential in ANTHROPIC_API_KEY

What it can do on your machine

Read from SKILL.md and the folder at commit 459c996. It shows what the files ask for, not the result of running them.

Tool permissions
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Runs code
No scripts in the folder and no shell commands in SKILL.md (its code samples are bash).
From the folder's file list and the shell code blocks in SKILL.md.
Network
No URLs in SKILL.md.
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names these keys or tokens, usually read from environment variables:
- ANTHROPIC_API_KEY
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Gui Automation loads about 1.4k tokens when it runs. Until then it costs about 78 tokens; SKILL.md has 354 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it: ~78
When it runs · the whole SKILL.md, loaded when a task matches: ~1.4k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from am-will/gooey-pi at commit 459c996, republished under its MIT licence (© am-will). 354 words, ~1,425 tokens.

Download SKILL.mdSave it as .claude/skills/gui-automation/SKILL.md (or your agent's skills folder).

name: gui-automation
description: Use when you need to visually interact with a GUI — test buttons, fill forms, verify visual layouts, fuzz web pages, automate user flows, take screenshots, or perform end-to-end QA on any application. Works on cloud VMs, Docker containers, local machines, and sandboxes. Install: pip install cua.

GUI Automation

CUA gives you eyes and hands on a real computer: see the screen, move the mouse, click, type, drag, and manage windows — like a human at the keyboard.

Use this skill for visual interaction that can't be done via shell or API.

Setup

bash

cua --version          # check install; if missing: pip install cua

# Connect to target (pick one)
cua do switch cloud my-vm
cua do switch docker my-container
cua do-host-consent && cua do switch host   # local machine (one-time consent)

ANTHROPIC_API_KEY is optional. With it, cua do snapshot returns an AI-annotated screen with element coordinates. Without it, use screenshot and read the image yourself.

Workflow

Look → Act → Verify — repeat until done, then share:

bash

cua do screenshot          # look
cua do click 450 280       # act
cua do screenshot          # verify
cua trajectory share       # share replay link with user

Re-screenshot after every UI change — coordinates go stale when the screen changes.

Scenarios

Click a button

bash

cua do screenshot
cua do click 450 280
cua do screenshot

Fill a form

bash

cua do screenshot
cua do click 400 200 && cua do type "Jane Doe"
cua do key tab            && cua do type "jane@example.com"
cua do key tab            && cua do type "SecureP@ss123"
cua do click 400 500
cua do screenshot

File upload dialog

bash

cua do click 350 400       # "Choose File"
cua do type "/home/user/report.pdf"
cua do key enter
cua do screenshot

Zoom in for precision clicks (host or small targets)

When clicking small or dense UI elements — especially on the host machine — zoom into the target window first. Coordinates become window-relative and screenshots show only that window, giving you higher effective resolution.

bash

cua do zoom "Google Chrome"   # crop to Chrome window; coords are now window-relative
cua do screenshot              # zoomed view — easier to locate small elements
cua do click 112 44            # precise click on a small tab or button
cua do screenshot              # verify
cua do unzoom                  # restore full-screen coords when done
cua do screenshot              # back to full desktop view

Use zoom any time click accuracy is uncertain. unzoom before switching windows or when you need to see the full desktop again.

Drag and drop

bash

cua do window ls               # list open windows
cua do drag 150 300 650 400    # source → destination
cua do screenshot

Fuzz a form

bash

cua do screenshot
cua do click 400 200
cua do type "<script>alert(1)</script>"
cua do key tab && cua do type "'; DROP TABLE users; --"
cua do key tab && cua do type "AAAAAAAAAAAAAAAAAAAAAAA"
cua do click 400 500
cua do screenshot              # check for errors, crashes, unexpected behavior

Trajectory

Every action is auto-recorded to ~/.cua/trajectories/{machine}/{session}/.

bash

cua trajectory share           # upload and get shareable HTTPS link (always do this at end)
cua trajectory ls              # list sessions
cua trajectory export          # generate HTML report
cua do --no-record click 100 200   # disable recording for a single action

Tell the user: "Here is the trajectory of my session: {url}"

Show full SKILL.md (167 more words)Show less

Quick Reference

Action	Command
Connect to target	`cua do switch <provider> [name]`
Screenshot	`cua do screenshot`
AI-annotated screen	`cua do snapshot ["instructions"]`
Click	`cua do click `<x>` `<y>` [left
Double-click	`cua do dclick <x> <y>`
Type text	`cua do type "text"`
Press key	`cua do key <key>`
Hotkey	`cua do hotkey <combo>` (e.g. `ctrl+c`)
Scroll	`cua do scroll <direction> [amount]`
Drag	`cua do drag <x1> <y1> <x2> <y2>`
Move cursor	`cua do move <x> <y>`
Shell command	`cua do shell "command"`
Open URL/file	`cua do open <url
List windows	`cua do window ls [app]`
Focus window	`cua do window focus <id>`
Zoom to window	`cua do zoom "App Name"`
Unzoom	`cua do unzoom`
Share trajectory	`cua trajectory share`

Providers

Provider	Example
`cloud`	`cua do switch cloud my-vm`
`cloudv2`	`cua do switch cloudv2 my-vm`
`docker`	`cua do switch docker my-container`
`lume`	`cua do switch lume my-vm`
`lumier`	`cua do switch lumier my-vm`
`winsandbox`	`cua do switch winsandbox`
`host`	`cua do switch host`

See references/command-reference.md for full argument syntax.

Files

Just SKILL.md in assets/skills/gui-automation of am-will/gooey-pi.

Open the folder on GitHubat commit 459c996

Compare with similar skills

Gui Automation next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Gui Automation compared with similar skills
Skill	Stars	Used in	Tokens	Auto-check	Licence	Repo updated
Gui Automation this skillam-will/gooey-pi	941	—	~1.4k	Automated safety check: Pass	MIT	29 days ago
Virtual DesktopLeoYeAI/openclaw-master-skills	2.2k	—	~4.5k	Automated safety check: Notes	MIT	2 mo ago
Cloudroutermanaflow-ai/manaflow	1.1k	—	~4.7k	Automated safety check: Pass	MIT	3 days ago
Borg Live Debugkaranhudia/borg-ui	1.7k	—	~1.4k	Automated safety check: Notes	AGPL-3.0	today
Oneclickvirtoneclickvirt/oneclickvirt	372	—	~1.1k	Automated safety check: Pass	GPL-3.0	7 days ago
Kei PR Readyrhoopr/kei	156	—	~1.2k	Automated safety check: Pass	MIT	today

Similar skills

Virtual Desktop
LeoYeAI/openclaw-master-skills
Full Computer Use for OpenClaw via kasmweb/chrome Docker sidecar.
2.2k GitHub stars~4.5k tokensUpdated 2 mo ago
Productivity & AutomationAuto-check: notes
Cloudrouter
manaflow-ai/manaflow
Manage cloud development sandboxes with cloudrouter. An agent skill from manaflow-ai/manaflow.
1.1k GitHub stars~4.7k tokensUpdated 3 days ago
Productivity & AutomationAuto-check passed
Borg Live Debug
karanhudia/borg-ui
Live Borg debugging by exec-ing into the borg-web-ui Docker container.
1.7k GitHub stars~1.4k tokensUpdated today
DevOps & CloudAuto-check: notes
Oneclickvirt
oneclickvirt/oneclickvirt
OneClickVirt operations skill for managing containers, virtual machines, provider nodes, health checks, and metrics through MCP.
372 GitHub stars~1.1k tokensUpdated 7 days ago
DevOps & CloudAuto-check passed
Kei PR Ready
rhoopr/kei
Validate a kei branch before review without publishing or changing it.
156 GitHub stars~1.2k tokensUpdated today
DevOps & CloudAuto-check passed
E2E
openathleteorg/openathlete
Run, debug or extend the OpenAthlete Playwright end-to-end tests, which exercise the production Docker images (API, worker, web, PostgreSQL, Redis) through the API and a real browser on desktop and…
101 GitHub stars~675 tokensUpdated today
Testing & QAAuto-check passed

Questions about Gui Automation

What does Gui Automation do?

A skill your agent uses when you need to visually interact with a GUI — test buttons, fill forms, verify visual layouts, fuzz web pages, automate user flows, take screenshots, or perform end-to-end…. Gui Automation is an agent skill from am-will/gooey-pi. Use when you need to visually interact with a GUI — test buttons, fill forms, verify visual layouts, fuzz web pages, automate user flows, take screenshots, or perform end-to-end QA on any application.

When should I use Gui Automation?

Gui Automation fits situations like: you need to visually interact with a GUI — test buttons; verify visual layouts; automate user flows; take screenshots.

How do I install Gui Automation in Claude Code?

Run `npx skills add am-will/gooey-pi --skill gui-automation -a claude-code`. Or copy the skill folder (assets/skills/gui-automation in am-will/gooey-pi) into .claude/skills/gui-automation in your project. Claude Code loads it when a task matches its description.

How do I install Gui Automation in Codex?

Run `npx skills add am-will/gooey-pi --skill gui-automation -a codex`. Or copy the skill folder (assets/skills/gui-automation in am-will/gooey-pi) into .agents/skills/gui-automation in your project. Codex loads it when a task matches its description.

Can I use Gui Automation in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add am-will/gooey-pi --skill gui-automation -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/gui-automation, .gemini/skills/gui-automation, .github/skills/gui-automation and .opencode/skills/gui-automation in your project.

What does Gui Automation need to run?

Going by SKILL.md and its folder, Gui Automation needs credentials named ANTHROPIC_API_KEY. Our summary lists: Python 3; Docker; A credential in ANTHROPIC_API_KEY.

Does Gui Automation access the network?

SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.

Is Gui Automation safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Gui Automation use?

Gui Automation is published under the MIT licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Gui Automation use?

About 1.4k tokens (SKILL.md is roughly 5.7k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Gui Automation?

Skills that share tags, products or a category with Gui Automation: Virtual Desktop (LeoYeAI/openclaw-master-skills, 2.2k stars), Cloudrouter (manaflow-ai/manaflow, 1.1k stars), Borg Live Debug (karanhudia/borg-ui, 1.7k stars) and Oneclickvirt (oneclickvirt/oneclickvirt, 372 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Gui Automation?

am-will (a GitHub user) maintains it in am-will/gooey-pi, which has 941 GitHub stars. The repository holds 8 skills in this directory. The repository was last updated on September 9, 2026.

Source: am-will/gooey-pi on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.

Install Gui Automation

Facts

At a glance

What it does

When your agent uses it

Example prompts

Requirements

What it can do on your machine

Tool permissions

Runs code

Network

Credentials

Context cost

Safety

SKILL.md

GUI Automation

Setup

Workflow

Scenarios

Click a button

Fill a form

File upload dialog

Zoom in for precision clicks (host or small targets)

Drag and drop

Fuzz a form

Trajectory

Quick Reference

Providers

Files

Compare with similar skills

Similar skills

More from am-will/gooey-pi

Related

Questions about Gui Automation