Agent Eval
ericrisco/rsc-harness
A skill your agent uses when measuring whether an LLM or agent system actually got better and gating merges on it: golden sets, fixing an inflated LLM-as-judge, scoring RAG (faithfulness, contextual…
Autonomous AI agent platform for building and deploying continuous agents.
$ npx skills add Orchestra-Research/AI-Research-SKILLs --skill autogpt-agents -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install Orchestra-Research/AI-Research-SKILLs autogpt-agents --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/Orchestra-Research/AI-Research-SKILLs.git skills-src && mkdir -p .claude/skills && cp -r skills-src/14-agents/autogpt .claude/skills/autogpt-agents && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "autogpt-agents" agent skill from https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/14-agents/autogpt into .claude/skills/autogpt-agents/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "autogpt-agents", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/14-agents/autogptType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add Orchestra-Research/AI-Research-SKILLs --skill autogpt-agents -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install Orchestra-Research/AI-Research-SKILLs autogpt-agents --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/Orchestra-Research/AI-Research-SKILLs.git skills-src && mkdir -p .agents/skills && cp -r skills-src/14-agents/autogpt .agents/skills/autogpt-agents && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "autogpt-agents" agent skill from https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/14-agents/autogpt into .agents/skills/autogpt-agents/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "autogpt-agents", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add Orchestra-Research/AI-Research-SKILLs --skill autogpt-agents -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install Orchestra-Research/AI-Research-SKILLs autogpt-agents --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/Orchestra-Research/AI-Research-SKILLs.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/14-agents/autogpt .cursor/skills/autogpt-agents && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "autogpt-agents" agent skill from https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/14-agents/autogpt into .cursor/skills/autogpt-agents/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "autogpt-agents", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/Orchestra-Research/AI-Research-SKILLs.git --path 14-agents/autogpt--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add Orchestra-Research/AI-Research-SKILLs --skill autogpt-agents -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install Orchestra-Research/AI-Research-SKILLs autogpt-agents --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/Orchestra-Research/AI-Research-SKILLs.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/14-agents/autogpt .gemini/skills/autogpt-agents && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "autogpt-agents" agent skill from https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/14-agents/autogpt into .gemini/skills/autogpt-agents/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "autogpt-agents", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install Orchestra-Research/AI-Research-SKILLs autogpt-agentsInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add Orchestra-Research/AI-Research-SKILLs --skill autogpt-agents -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/Orchestra-Research/AI-Research-SKILLs.git skills-src && mkdir -p .github/skills && cp -r skills-src/14-agents/autogpt .github/skills/autogpt-agents && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "autogpt-agents" agent skill from https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/14-agents/autogpt into .github/skills/autogpt-agents/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "autogpt-agents", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add Orchestra-Research/AI-Research-SKILLs --skill autogpt-agents -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install Orchestra-Research/AI-Research-SKILLs autogpt-agents --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/Orchestra-Research/AI-Research-SKILLs.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/14-agents/autogpt .opencode/skills/autogpt-agents && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "autogpt-agents" agent skill from https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/14-agents/autogpt into .opencode/skills/autogpt-agents/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "autogpt-agents", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
autogpt-agentsAutonomous AI agent platform for building and deploying continuous agents.
Autogpt Agents is an agent skill from Orchestra-Research/AI-Research-SKILLs. Autonomous AI agent platform for building and deploying continuous agents. Use when creating visual workflow agents, deploying persistent autonomous agents, or building complex multi-step AI automation systems.
Its SKILL.md is about 2.3k tokens, which your agent loads only when the skill is triggered. The skill folder holds 3 other files, including reference files (for example `references/advanced-usage.md` and `references/troubleshooting.md`).
It sits in AI & LLM Engineering, covering Autonomous loops and Building AI agents. The repository describes itself as: Comprehensive open-source library of AI research and engineering skills for any AI model. Package the skills and your claude code/codex/gemini agent will be an AI research agent… The licence is MIT.
5 steps, taken from the first numbered list in SKILL.md.
Read from SKILL.md and the folder at commit 773a529. It shows what the files ask for, not the result of running them.
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Shell commands in SKILL.md call:
dockernpmpoetrygitFrom the folder's file list and the shell code blocks in SKILL.md.
Hosts in commands or code, which the agent is likely to contact:
github.comAlso links to:
docs.agpt.codiscord.ggFrom URLs in SKILL.md, links to its own repository left out.
Names these keys or tokens, usually read from environment variables:
ENCRYPTION_KEYFrom names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Autogpt Agents loads about 2.3k tokens when it runs, and up to ~7.5k if it reads all its reference files. Until then it costs about 56 tokens; SKILL.md has 558 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
The automated check noted patterns worth knowing about, such as sudo or a known installer.
cp .env.example .envcp .env.example .envAutomated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.
The full file from Orchestra-Research/AI-Research-SKILLs at commit 773a529, republished under its MIT licence (© Orchestra-Research). 558 words, ~2,300 tokens.
.claude/skills/autogpt-agents/SKILL.md (or your agent's skills folder). This skill also uses 2 other files; get the full folder from GitHub.Comprehensive platform for building, deploying, and managing continuous AI agents through a visual interface or development toolkit.
Use AutoGPT when:
Key features:
Use alternatives instead:
# Clone repository
git clone https://github.com/Significant-Gravitas/AutoGPT.git
cd AutoGPT/autogpt_platform
# Copy environment file
cp .env.example .env
# Start backend services
docker compose up -d --build
# Start frontend (in separate terminal)
cd frontend
cp .env.example .env
npm install
npm run devAutoGPT has two main systems:
Agents are represented as graphs containing nodes connected by links:
Graph (Agent)
├── Node (Input)
│ └── Block (AgentInputBlock)
├── Node (Process)
│ └── Block (LLMBlock)
├── Node (Decision)
│ └── Block (SmartDecisionMaker)
└── Node (Output)
└── Block (AgentOutputBlock)Blocks are reusable functional components:
| Block Type | Purpose |
|---|---|
INPUT | Agent entry points |
OUTPUT | Agent outputs |
AI | LLM calls, text generation |
WEBHOOK | External triggers |
STANDARD | General operations |
AGENT | Nested agent execution |
User/Trigger → Graph Execution → Node Execution → Block.execute()
↓ ↓ ↓
Inputs Queue System Output YieldsAI Blocks:
AITextGeneratorBlock - Generate text with LLMsAIConversationBlock - Multi-turn conversationsSmartDecisionMakerBlock - Conditional logicIntegration Blocks:
Control Blocks:
Manual execution:
POST /api/v1/graphs/{graph_id}/execute
Content-Type: application/json
{
"inputs": {
"input_name": "value"
}
}Webhook trigger:
POST /api/v1/webhooks/{webhook_id}
Content-Type: application/json
{
"data": "webhook payload"
}Scheduled execution:
{
"schedule": "0 */2 * * *",
"graph_id": "graph-uuid",
"inputs": {}
}WebSocket updates:
const ws = new WebSocket('ws://localhost:8001/ws');
ws.onmessage = (event) => {
const update = JSON.parse(event.data);
console.log(`Node ${update.node_id}: ${update.status}`);
};REST API polling:
GET /api/v1/executions/{execution_id}# Setup forge environment
cd classic
./run setup
# Create new agent from template
./run forge create my-agent
# Start agent server
./run forge start my-agentmy-agent/
├── agent.py # Main agent logic
├── abilities/ # Custom abilities
│ ├── __init__.py
│ └── custom.py
├── prompts/ # Prompt templates
└── config.yaml # Agent configurationfrom forge import Ability, ability
@ability(
name="custom_search",
description="Search for information",
parameters={
"query": {"type": "string", "description": "Search query"}
}
)
def custom_search(query: str) -> str:
"""Custom search ability."""
# Implement search logic
result = perform_search(query)
return result# Run all benchmarks
./run benchmark
# Run specific category
./run benchmark --category coding
# Run with specific agent
./run benchmark --agent my-agentBenchmarks use recorded HTTP responses for reproducibility:
# Record new cassettes
./run benchmark --record
# Run with existing cassettes
./run benchmark --playbackBlocks automatically access user credentials:
class MyLLMBlock(Block):
def execute(self, inputs):
# Credentials are injected by the system
credentials = self.get_credentials("openai")
client = OpenAI(api_key=credentials.api_key)
# ...| Provider | Auth Type | Use Cases |
|---|---|---|
| OpenAI | API Key | LLM, embeddings |
| Anthropic | API Key | Claude models |
| GitHub | OAuth | Code, repos |
| OAuth | Drive, Gmail, Calendar | |
| Discord | Bot Token | Messaging |
| Notion | OAuth | Documents |
# docker-compose.prod.yml
services:
rest_server:
image: autogpt/platform-backend
environment:
- DATABASE_URL=postgresql://...
- REDIS_URL=redis://redis:6379
ports:
- "8006:8006"
executor:
image: autogpt/platform-backend
command: poetry run executor
frontend:
image: autogpt/platform-frontend
ports:
- "3000:3000"| Variable | Purpose |
|---|---|
DATABASE_URL | PostgreSQL connection |
REDIS_URL | Redis connection |
RABBITMQ_URL | RabbitMQ connection |
ENCRYPTION_KEY | Credential encryption |
SUPABASE_URL | Authentication |
cd autogpt_platform/backend
poetry run cli gen-encrypt-keyServices not starting:
# Check container status
docker compose ps
# View logs
docker compose logs rest_server
# Restart services
docker compose restartDatabase connection issues:
# Run migrations
cd backend
poetry run prisma migrate deployAgent execution stuck:
# Check RabbitMQ queue
# Visit http://localhost:15672 (guest/guest)
# Clear stuck executions
docker compose restart executor© Orchestra-Research, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
SKILL.md and 2 other files (references) in 14-agents/autogpt of Orchestra-Research/AI-Research-SKILLs.
Open the folder on GitHubat commit 773a529
We found 2 copies of this SKILL.md (exact, near-identical or edited) in other folders, from 2 other GitHub owners. This page covers the copy in Orchestra-Research/AI-Research-SKILLs, which our catalogue first saw on October 7, 2026.
Autogpt Agents next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Autogpt Agents this skillOrchestra-Research/AI-Research-SKILLs | 13k | 2 repos | ~2.3k | Automated safety check: Notes | MIT | |
| Agent Evalericrisco/rsc-harness | 180 | — | ~3.2k | Automated safety check: Pass | MIT | |
| AWS Harnesshoodini/ai-agents-skills | 282 | — | ~4.2k | Automated safety check: Notes | None | |
| Prompt Engineeringericrisco/rsc-harness | 180 | — | ~2.4k | Automated safety check: Pass | MIT | |
| Chatbotericrisco/rsc-harness | 180 | — | ~3.3k | Automated safety check: Pass | MIT | |
| Swarms Multi-Agent Frameworkkyegomez/swarms | 7.2k | — | ~5.5k | Automated safety check: Pass | Apache-2.0 |
ericrisco/rsc-harness
A skill your agent uses when measuring whether an LLM or agent system actually got better and gating merges on it: golden sets, fixing an inflated LLM-as-judge, scoring RAG (faithfulness, contextual…
hoodini/ai-agents-skills
Build a new AI agent on AWS and deploy it easily, OR wrap and deploy an agent you already have, using the Amazon Bedrock AgentCore harness.
ericrisco/rsc-harness
A skill your agent uses when one prompt must give the same right answer across reruns, models, and pasted-in hostile input: forcing a fixed schema, picking the few-shot set, ordering the prompt…
ericrisco/rsc-harness
A skill your agent uses when a support or sales bot on a live website must behave: persona/system prompt, grounding so it cannot invent prices or policy, jailbreak and injection defense, the human…
kyegomez/swarms
Teaches the Swarms Python framework: the Agent class, tools, loops, memory and multi-agent structures such as sequential, concurrent and graph workflows.
UiPath/skills
UiPath Coded Functions — deterministic Python or TypeScript/JavaScript units built with the uip function CLI (new -l py|ts|js, init, serve, run, pack, publish); the functions map in uipath.json…
Orchestra-Research/AI-Research-SKILLs
Generates music from text descriptions with MusicGen and sound effects with AudioGen, using Meta's AudioCraft PyTorch library with melody and style conditioning.
Orchestra-Research/AI-Research-SKILLs
Runs lm-evaluation-harness to benchmark language models on academic suites such as MMLU, GSM8K and HumanEval, compare models and track training checkpoints.
Orchestra-Research/AI-Research-SKILLs
Guide to using Meta's Segment Anything Model for zero-shot image segmentation with point, box or mask prompts, or automatic mask generation.
Orchestra-Research/AI-Research-SKILLs
Shows how to store documents and embeddings in Chroma, query them by similarity with metadata filters, and persist them to disk for RAG and semantic search projects.
Orchestra-Research/AI-Research-SKILLs
Explains OpenAI's CLIP model for zero-shot image classification, image-text similarity, semantic image search and content moderation, with install steps and code patterns.
Orchestra-Research/AI-Research-SKILLs
Transcribes audio with OpenAI's Whisper: 99 languages, translation to English, language detection, six model sizes and word-level timestamps, from Python or the CLI.
Autonomous AI agent platform for building and deploying continuous agents. Autogpt Agents is an agent skill from Orchestra-Research/AI-Research-SKILLs. Autonomous AI agent platform for building and deploying continuous agents.
Autogpt Agents fits situations like: creating visual workflow agents; deploying persistent autonomous agents; building complex multi-step AI automation systems.
Run `npx skills add Orchestra-Research/AI-Research-SKILLs --skill autogpt-agents -a claude-code`. Or copy the skill folder (14-agents/autogpt in Orchestra-Research/AI-Research-SKILLs) into .claude/skills/autogpt-agents in your project. Claude Code loads it when a task matches its description.
Run `npx skills add Orchestra-Research/AI-Research-SKILLs --skill autogpt-agents -a codex`. Or copy the skill folder (14-agents/autogpt in Orchestra-Research/AI-Research-SKILLs) into .agents/skills/autogpt-agents in your project. Codex loads it when a task matches its description.
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add Orchestra-Research/AI-Research-SKILLs --skill autogpt-agents -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/autogpt-agents, .gemini/skills/autogpt-agents, .github/skills/autogpt-agents and .opencode/skills/autogpt-agents in your project.
Going by SKILL.md and its folder, Autogpt Agents needs the command-line tools its instructions call (docker, npm, poetry and git) and credentials named ENCRYPTION_KEY. Our summary lists: Python 3; Node.js; Docker; A credential in ENCRYPTION_KEY.
SKILL.md names 3 domains. In commands or code: github.com; the agent is likely to contact it when it follows the instructions. As links in the text: docs.agpt.co and discord.gg. This is read from the text; nothing was executed.
Our automated static check of SKILL.md found notes only (mentions a .env file), nothing it rates as a warning. It is not a guarantee. Review the folder before installing.
Autogpt Agents is published under the MIT licence (declared in SKILL.md). It allows redistribution, so the full SKILL.md is shown on this page.
About 2.3k tokens (SKILL.md is roughly 9.2k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 5.2k tokens, read only when the agent opens those files.
Skills that share tags, products or a category with Autogpt Agents: Agent Eval (ericrisco/rsc-harness, 180 stars), AWS Harness (hoodini/ai-agents-skills, 282 stars), Prompt Engineering (ericrisco/rsc-harness, 180 stars) and Chatbot (ericrisco/rsc-harness, 180 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
Orchestra-Research (a GitHub organization) maintains it in Orchestra-Research/AI-Research-SKILLs, which has 13,405 GitHub stars. The repository holds 96 skills in this directory. The repository was last updated on June 16, 2026.
Source: Orchestra-Research/AI-Research-SKILLs on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.