Cherry Studio Regression Tests

What it does

When your agent uses it

• Running the full regression suite before a release
• Validating a development branch on macOS and Windows runners
• Adding or changing a regression case in tests/e2e/regression
• Running a named cherry-regression-test task

Example prompts

• “Run the Cherry Studio regression workflow against the release candidate tag.”
• “Add a regression case for knowledge base import following the manifest pattern.”
• “The Windows run failed in the agents phase. Pull the report and tell me what broke.”

Requirements

• A GitHub repository with runners for macOS and Windows
• Repository variables and secrets for the test providers

Workflow steps

8 steps, taken from the first numbered list in SKILL.md.

1. 1Resolves a trusted branch or release tag.
2. 2Initializes an isolated directory under the GitHub runner temporary folder.
3. 3Installs the application and the code tools under test.
4. 4Launches one owned Electron process with CDP enabled.
5. 5Runs the ten files in tests/e2e/regression/ from simple to complex.
6. 6Continues after a failed phase so later results are still collected.
7. 7Produces English platform and aggregate reports, then enforces the verdict.
8. 8Stops only the Electron process recorded in the isolated run directory.

What it can do on your machine

Read from SKILL.md and the folder at commit c97c23c. It shows what the files ask for, not the result of running them.

• Tool permissions

  Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

  From allowed-tools in the SKILL.md frontmatter.

• Runs code

  Shell commands in SKILL.md call:

  • pnpm

  From the folder's file list and the shell code blocks in SKILL.md.

• Network

  No URLs in SKILL.md. Its commands use pnpm, which can reach the network depending on how they are called.

  From URLs in SKILL.md, links to its own repository left out.

• Credentials

  Names these keys or tokens, usually read from environment variables:

  • CHERRY_TEST_CUSTOM_PROVIDER_API_KEY
  • CHERRY_TEST_CUSTOM_PROVIDER_EMBEDDING_API_KEY
  • CHERRY_TEST_CHERRYIN_PASSWORD

  From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Cherry Studio Regression Tests loads about 1.2k tokens when it runs. Until then it costs about 75 tokens; SKILL.md has 476 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it

~75

When it runs · the whole SKILL.md, loaded when a task matches

~1.2k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from CherryHQ/cherry-studio at commit c97c23c, republished under its AGPL-3.0 licence (© CherryHQ). 476 words, ~1,189 tokens.

name

cherry-regression-test

description

Run Cherry Studio critical-path system regression tasks through the repository-owned Playwright E2E workflow. Use for full regression, release acceptance, development-branch system validation, or a named cherry-regression-test task on GitHub-hosted macOS and Windows runners.

test(...caseDefinition('S-01'), async ({ mainWindow }) => {
  // Assert the user-visible outcome.
})

pnpm exec tsx scripts/e2e/regression/cli.ts run-phase \
  --run-dir /absolute/run-directory --phase 02-basic-features

CHERRY_TEST_RUN_DIR=/tmp/cherry-regression-list pnpm test:e2e:regression --list

pnpm exec vitest run --project scripts scripts/e2e/regression
CHERRY_TEST_RUN_DIR=/tmp/cherry-regression-list \
  pnpm test:e2e:regression --list
pnpm typecheck:e2e
pnpm test:lint

© CherryHQ, AGPL-3.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in .agents/skills/cherry-regression-test of CherryHQ/cherry-studio.

Open the folder on GitHubat commit c97c23c