Aidd Riteway AI

What it does

When your agent uses it

• Writing prompt evals
• Creating .sudo test files
• Testing agent skills that use tools such as gh

Example prompts

• “Use the aidd-riteway-ai skill to teach agents how to write correct riteway ai prompt evals (.sudo files) for multi-step flows that involve tool calls”
• “/aidd-riteway-ai”

Requirements

• Node.js
• Compatibility (from SKILL.md): Requires riteway >=9 with the `riteway ai` subcommand available.

Workflow steps

7 steps, taken from the first numbered list in SKILL.md.

1. 1Read the skill under test and its functional requirements
2. 2Identify the discrete steps in the skill's flow
3. 3Create one .sudo eval file per step (Rule 1), placed in ai-evals//
4. 4For each file, write the userPrompt — include mock tool preambles for unit evals (Rule 2), assert tool calls for step 1 (Rule 3), supply…
5. 5Write assertions derived strictly from functional requirements in Given X, should Y format (Rule 7)
6. 6Create small, single-condition fixture files as needed (Rule 6)
7. 7Verify against the Eval Authoring Checklist below

What it can do on your machine

Read from SKILL.md and the folder at commit 9a7c8e3. It shows what the files ask for, not the result of running them.

• Tool permissions

  Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

  From allowed-tools in the SKILL.md frontmatter.

• Runs code

  Ships script files (JavaScript), which the agent can run.

  From the folder's file list and the shell code blocks in SKILL.md.

• Network

  No URLs in SKILL.md.

  From URLs in SKILL.md, links to its own repository left out.

• Credentials

  Names no API keys, tokens, secrets or passwords.

  From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

• Compatibility

  Requires riteway >=9 with the `riteway ai` subcommand available.

  From compatibility in the SKILL.md frontmatter.

Context cost

Aidd Riteway AI loads about 1.6k tokens when it runs. Until then it costs about 68 tokens; SKILL.md has 665 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it

~68

When it runs · the whole SKILL.md, loaded when a task matches

~1.6k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from paralleldrive/aidd at commit 9a7c8e3, republished under its MIT licence (© paralleldrive). 665 words, ~1,636 tokens.

name

aidd-riteway-ai

description

Teaches agents how to write correct riteway ai prompt evals (.sudo files) for multi-step flows that involve tool calls. Use when writing prompt evals, creating .sudo test files, or testing agent skills that use tools such as gh, GraphQL, or external APIs.

compatibility

Requires riteway >=9 with the `riteway ai` subcommand available.

import 'ai/skills/<skill-name>/SKILL.md'

userPrompt = """
<prompt sent to the agent under test>
"""

- Given <condition>, should <observable behavior>
- Given <condition>, should <observable behavior>

ai-evals/<skill-name>/step-1-<description>-test.sudo
ai-evals/<skill-name>/step-2-<description>-test.sudo

You have the following mock tools available. Use them instead of real gh or GraphQL calls:

mock gh pr view => returns:
  title: My PR
  branch: feature/foo
  base: main

mock gh api (list review threads) => returns:
  [{ id: "T_01", resolved: false, body: "..." }]

userPrompt = """
You have mock tools available. Use them instead of real API calls.
Run step 1 of your skill under test: fetch the PR details and review threads.
"""

- Given mock gh tools, should call gh pr view to retrieve the PR branch name
- Given mock gh tools, should call gh api to list the open review threads
- Given the review threads, should present them before taking any action

# ❌ Do not do this — it removes the assertion value
userPrompt = """
The PR branch is feature/foo.
The review threads are: [...]
Now generate delegation prompts.
"""

userPrompt = """
You have mock tools available. Use them instead of real calls.

Triage is complete. The following issues remain unresolved:

Issue 1 (thread ID: T_01):
  File: src/utils.js, line 5
  "add() subtracts instead of adding"

Generate delegation prompts for the remaining issues.
"""

ai-evals/<skill-name>/step-1-<description>-e2e.test.sudo

ai-evals/<skill-name>/fixtures/<filename>

export const add = (a, b) => a - b; // bug: subtracts instead of adds

- Given <condition>, should <observable behavior>

© paralleldrive, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 2 other files in ai/skills/aidd-riteway-ai of paralleldrive/aidd.

• SKILL.md
• README.md
• riteway-ai.test.js

Open the folder on GitHubat commit 9a7c8e3