Launch an autonomous THINK→EXECUTE→REFLECT experiment loop on a GPU project

Apache-2.0Auto-check passedAI & LLM Engineering

Install Auto Experiment

skills CLI
$ npx skills add Xiangyue-Zhang/auto-deep-researcher-24x7 --skill auto-experiment -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install Xiangyue-Zhang/auto-deep-researcher-24x7 auto-experiment --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/Xiangyue-Zhang/auto-deep-researcher-24x7.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/auto-experiment .claude/skills/auto-experiment && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
auto-experiment
GitHub stars
1.3k
Token cost
~1.4k tokens
SKILL.md length
428 words
Files
2
Skills in repo
8
Repo updated
First seen
Licence
Apache-2.0

At a glance

Launch an autonomous THINK→EXECUTE→REFLECT experiment loop on a GPU project

  • Works in 3 steps: THINK → EXECUTE → REFLECT
  • Tasks that involve Deep learning
  • SKILL.md covers What This Does, Usage, Prerequisites and Workflow Details, plus 3 more sections
  • Calls claude

What it does

Auto Experiment is an agent skill from Xiangyue-Zhang/auto-deep-researcher-24x7. Launch an autonomous THINK→EXECUTE→REFLECT experiment loop on a GPU project

Its SKILL.md is about 1.4k tokens, which your agent loads only when the skill is triggered. The skill folder holds 2 other files (for example `agents/openai.yaml`).

It sits in AI & LLM Engineering, covering Deep learning. The repository describes itself as: 🔥 An autonomous AI agent that runs your deep learning experiments 24/7 while you sleep. Zero-cost monitoring, Leader-Worker architecture, constant-size memory. The licence is Apache-2.0.

When your agent uses it

  • Tasks that involve Deep learning

Example prompts

  • “/auto-experiment”

Workflow steps

3 steps, taken from the step headings in SKILL.md.

  1. THINK
  2. EXECUTE
  3. REFLECT

What it can do on your machine

Read from SKILL.md and the folder at commit dbf3df8. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Shell commands in SKILL.md call:

    • claude

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    No URLs in SKILL.md.

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Auto Experiment loads about 1.4k tokens when it runs. Until then it costs about 23 tokens; SKILL.md has 428 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~23
When it runs · the whole SKILL.md, loaded when a task matches
~1.4k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from Xiangyue-Zhang/auto-deep-researcher-24x7 at commit dbf3df8, republished under its Apache-2.0 licence (© Xiangyue-Zhang). 428 words, ~1,396 tokens.

Download SKILL.mdSave it as .claude/skills/auto-experiment/SKILL.md (or your agent's skills folder). This skill also uses 1 other file; get the full folder from GitHub.
name
auto-experiment
description
Launch an autonomous THINK→EXECUTE→REFLECT experiment loop on a GPU project

auto-experiment

Launch an autonomous experiment agent that runs your deep learning experiments 24/7.

What This Does

This skill starts a THINK → EXECUTE → REFLECT loop that:

  1. Reads your PROJECT_BRIEF.md to understand the research goal
  2. Analyzes previous results in MEMORY_LOG.md
  3. Plans the next experiment (hypothesis + success criteria)
  4. Implements code changes and runs a mandatory dry-run
  5. Launches GPU training via nohup (tracks PID)
  6. Monitors at zero LLM cost (only kill -0 PID + tail log + nvidia-smi)
  7. Wakes up when training finishes to analyze results
  8. Updates memory and decides: iterate, pivot, or report
  9. Repeats

Usage

Claude Code: /auto-experiment
Claude Code: /auto-experiment --project /path/to/my_project --gpu 0
Claude Code: /auto-experiment --project . --max-cycles 5
Codex: $auto-experiment

Prerequisites

The project directory must contain:

PROJECT_BRIEF.md (required)

A frozen reference describing your research goal. Example:

markdown
# Goal
Train a ViT-B/16 on ImageNet to reach 78%+ top-1 accuracy.

# Codebase
- Training: train.py
- Config: configs/vit_base.yaml
- Data: /data/imagenet/

# Constraints
- GPU 0-3 available (use DDP)
- Max 90 epochs per run
- Report val accuracy after each run

# Current Best
- ResNet-50 baseline: 76.1%
config.yaml (optional)

Override default agent settings:

yaml
agent:
  provider: "anthropic"    # or "openai" / "claude_cli" / "codex_cli"
  model: "claude-sonnet-4-6"
  base_url: ""             # optional compatible endpoint override
  api_key_env: ""          # optional custom key env var
  auth_token_env: ""       # optional custom bearer token env var
  max_cycles: -1          # -1 = unlimited
  max_steps_per_cycle: 3  # max sub-agent dispatches per cycle
  cooldown_interval: 300  # 5 min smart polling

memory:
  brief_max_chars: 3000
  log_max_chars: 2000

monitor:
  poll_interval: 900      # check every 15 min during training
  zero_llm: true

experiment:
  mandatory_dry_run: true

If the user wants a compatible API endpoint instead of the official Anthropic or OpenAI API, keep the same provider values and set base_url plus a custom api_key_env. Do not invent provider names like qwen or glm.

Optional remote execution over SSH:

yaml
execution:
  mode: "ssh"
  ssh_host: "user@server"
  remote_workspace: "/home/user/my_project/workspace"
  remote_python: "python3"

In SSH mode, the controller state stays local (PROJECT_BRIEF.md, workspace/MEMORY_LOG.md, workspace/HUMAN_DIRECTIVE.md, state.json), while code edits, shell commands, training, log tailing, PID checks, and GPU queries run on the configured remote host.

Workflow Details

Phase 1: THINK
  • Read PROJECT_BRIEF.md (frozen, max 3000 chars)
  • Read MEMORY_LOG.md (rolling, auto-compacted)
  • Check for HUMAN_DIRECTIVE.md (highest priority, auto-archived after reading)
  • Analyze: What's the current best? What hasn't been tried? What's most promising?
  • Output: experiment plan with hypothesis and success criteria
Show full SKILL.md (191 more words)Show less
Phase 2: EXECUTE
  • Dispatch to Code Agent (5 tools: run_shell, launch_experiment, write_file, read_file, list_files)
  • Code Agent implements changes
  • Mandatory dry-run (2-step verify, abort if fails)
  • Launch training via nohup, capture PID
  • Enter zero-cost monitoring loop:
    • backend PID check — is process alive?
    • backend nvidia-smi — GPU utilization
    • backend tail -50 logfile — latest training output
    • Zero LLM API calls during this phase
Phase 3: REFLECT
  • Parse training logs for metrics (loss, accuracy, FGD, FID, etc.)
  • Compare against previous best
  • Log milestone if improved (auto-compacted at 1200 chars)
  • Log decision (rolling last 15 entries)
  • Decide: try another config / pivot direction / generate report
Human Override (anytime)
bash
# Drop a directive file — agent reads it next cycle with highest priority
echo "Try learning rate 1e-5 with cosine schedule" > workspace/HUMAN_DIRECTIVE.md

Memory System

Two-Tier, constant size (~5K chars / ~1500 tokens), no matter how long the agent runs:

TierFileContentCap
1PROJECT_BRIEF.mdFrozen project reference3,000 chars
2MEMORY_LOG.mdKey Results + Recent Decisions2,000 chars

Auto-compaction rules:

  • Key Results: oldest dropped when section > 1,200 chars
  • Recent Decisions: only last 15 entries kept
  • Total log hard-capped at 2,000 chars

Cost

PhaseDurationLLM Cost
THINK5-10 min~$0.05
EXECUTE (training)hours/days$0.00
REFLECT5-10 min~$0.03
24h cycle total~$0.08

Example Output

After a few cycles, your workspace/MEMORY_LOG.md will look like:

markdown
# Memory Log

## Key Results
[04-07 14:30] Exp001: ResNet-50 baseline, lr=0.1, acc=76.1%
[04-07 22:15] Exp002: ViT-B/16, lr=1e-3, acc=74.8% (underperforming, lr too high)
[04-08 06:00] Exp003: ViT-B/16, lr=3e-4 + cosine, acc=77.9% (new best!)
[04-08 14:45] Exp004: ViT-B/16, lr=3e-4 + cosine + mixup, acc=78.3% (target reached!)

## Recent Decisions
[04-07 14:30] Start with ResNet-50 baseline to establish reference
[04-07 22:15] ViT lr=1e-3 too high, try 3e-4 next
[04-08 06:00] Cosine schedule helped significantly, try adding regularization
[04-08 14:45] Target reached! Generate final report.

© Xiangyue-Zhang, Apache-2.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 1 other file in skills/auto-experiment of Xiangyue-Zhang/auto-deep-researcher-24x7.

  • SKILL.md
  • agents/openai.yaml

Open the folder on GitHubat commit dbf3df8

Compare with similar skills

Auto Experiment next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Auto Experiment compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Auto Experiment this skillXiangyue-Zhang/auto-deep-researcher-24x71.3k—~1.4kAutomated safety check: PassApache-2.0
Add Uint Supportpytorch/pytorch104k2 repos~2.3kAutomated safety check: PassCustom licence
Segment Anything Model GuideOrchestra-Research/AI-Research-SKILLs13k8 repos~3.3kAutomated safety check: PassMIT
Add Oponnx/onnx22k—~1.2kAutomated safety check: PassApache-2.0
CLIP Image-Text MatchingOrchestra-Research/AI-Research-SKILLs13k7 repos~1.7kAutomated safety check: PassMIT
Add Function Bodyonnx/onnx22k—~1.1kAutomated safety check: PassApache-2.0

Similar skills

  • Add Uint Support

    pytorch/pytorch

    Add unsigned integer (uint) type support to PyTorch operators by updating ATDISPATCH macros.

    104k GitHub starsUsed in 2 repos~2.3k tokens
    AI & LLM EngineeringAuto-check passed
  • Segment Anything Model Guide

    Orchestra-Research/AI-Research-SKILLs

    Guide to using Meta's Segment Anything Model for zero-shot image segmentation with point, box or mask prompts, or automatic mask generation.

    13k GitHub starsUsed in 8 repos~3.3k tokens
    AI & LLM EngineeringAuto-check passed
  • Add Op

    onnx/onnx

    Add a new ONNX operator or update an existing operator to a new opset version.

    22k GitHub stars~1.2k tokensUpdated today
    AI & LLM EngineeringAuto-check passed
  • CLIP Image-Text Matching

    Orchestra-Research/AI-Research-SKILLs

    Explains OpenAI's CLIP model for zero-shot image classification, image-text similarity, semantic image search and content moderation, with install steps and code patterns.

    13k GitHub starsUsed in 7 repos~1.7k tokens
    AI & LLM EngineeringAuto-check passed
  • Add a function body definition to an ONNX operator, defining how it decomposes into simpler ops.

    22k GitHub stars~1.1k tokensUpdated today
    AI & LLM EngineeringAuto-check passed
  • Paddle Design Distributed

    PaddlePaddle/Paddle

    A skill your agent uses when working with Paddle's distributed training system: understanding parallelism strategies (DP, ZeRO, TP, PP, SP), semi-automatic parallel with ProcessMesh + shardtensor…

    24k GitHub stars~660 tokensUpdated yesterday
    AI & LLM EngineeringAuto-check passed

More from Xiangyue-Zhang/auto-deep-researcher-24x7

All 8 skills in this repo
  • Conf Search

    Xiangyue-Zhang/auto-deep-researcher-24x7

    Search papers from top AI/ML conferences. An agent skill from Xiangyue-Zhang/auto-deep-researcher-24x7.

    1.3k GitHub stars~177 tokensUpdated 4 mo ago
    Auto-check passed
  • Daily Papers

    Xiangyue-Zhang/auto-deep-researcher-24x7

    Daily arXiv paper recommendations with automatic deduplication

    1.3k GitHub stars~309 tokensUpdated 4 mo ago
    Auto-check passed
  • Experiment Status

    Xiangyue-Zhang/auto-deep-researcher-24x7

    Check status of running autonomous experiment loops. An agent skill from Xiangyue-Zhang/auto-deep-researcher-24x7.

    1.3k GitHub stars~381 tokensUpdated 4 mo ago
    Auto-check passed
  • GPU Monitor

    Xiangyue-Zhang/auto-deep-researcher-24x7

    Check GPU status, running experiments, and available resources

    1.3k GitHub stars~280 tokensUpdated 4 mo ago
    Auto-check passed
  • Obsidian Sync

    Xiangyue-Zhang/auto-deep-researcher-24x7

    Refresh Obsidian dashboard and daily notes from current experiment state

    1.3k GitHub stars~247 tokensUpdated 4 mo ago
    Auto-check passed
  • Paper Analyze

    Xiangyue-Zhang/auto-deep-researcher-24x7

    Deep analysis of a single paper with figure extraction from arXiv source

    1.3k GitHub stars~306 tokensUpdated 4 mo ago
    Auto-check passed

Questions about Auto Experiment

What does Auto Experiment do?

Launch an autonomous THINK→EXECUTE→REFLECT experiment loop on a GPU project. Auto Experiment is an agent skill from Xiangyue-Zhang/auto-deep-researcher-24x7.

When should I use Auto Experiment?

Auto Experiment fits situations like: tasks that involve Deep learning.

How do I install Auto Experiment in Claude Code?

Run `npx skills add Xiangyue-Zhang/auto-deep-researcher-24x7 --skill auto-experiment -a claude-code`. Or copy the skill folder (skills/auto-experiment in Xiangyue-Zhang/auto-deep-researcher-24x7) into .claude/skills/auto-experiment in your project. Claude Code loads it when a task matches its description.

How do I install Auto Experiment in Codex?

Run `npx skills add Xiangyue-Zhang/auto-deep-researcher-24x7 --skill auto-experiment -a codex`. Or copy the skill folder (skills/auto-experiment in Xiangyue-Zhang/auto-deep-researcher-24x7) into .agents/skills/auto-experiment in your project. Codex loads it when a task matches its description.

Can I use Auto Experiment in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add Xiangyue-Zhang/auto-deep-researcher-24x7 --skill auto-experiment -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/auto-experiment, .gemini/skills/auto-experiment, .github/skills/auto-experiment and .opencode/skills/auto-experiment in your project.

What does Auto Experiment need to run?

Going by SKILL.md and its folder, Auto Experiment needs the command-line tools its instructions call (claude).

Does Auto Experiment access the network?

SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.

Is Auto Experiment safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Auto Experiment use?

Auto Experiment is published under the Apache-2.0 licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Auto Experiment use?

About 1.4k tokens (SKILL.md is roughly 5.6k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Auto Experiment?

Skills that share tags, products or a category with Auto Experiment: Add Uint Support (pytorch/pytorch, 104k stars), Segment Anything Model Guide (Orchestra-Research/AI-Research-SKILLs, 13k stars), Add Op (onnx/onnx, 22k stars) and CLIP Image-Text Matching (Orchestra-Research/AI-Research-SKILLs, 13k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Auto Experiment?

Xiangyue-Zhang (a GitHub user) maintains it in Xiangyue-Zhang/auto-deep-researcher-24x7, which has 1,296 GitHub stars. The repository holds 8 skills in this directory. The repository was last updated on June 3, 2026.

Source: Xiangyue-Zhang/auto-deep-researcher-24x7 on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.