Strudel Music

⚠️ Legal Notice: This tool processes audio you provide. You are responsible for ensuring you have the rights to use the source material. The authors make no claims about fair use, copyright, or derivative works regarding your use of this tool with copyrighted material.

Strudel Music 🎵

Compose, render, deconstruct, and remix music using code. Takes natural language prompts → writes Strudel patterns → renders offline through real Web Audio synthesis → posts audio or streams to Discord VC (via the OpenClaw gateway — no separate credentials needed). Can also reverse-engineer any audio track into stems, samples, and generative programs.

New here? Read docs/ONBOARDING.md for a ground-up introduction.

⚠️ SESSION SAFETY — READ THIS FIRST

Rendering MUST run as a sub-agent or background process, never inline in your main session.

The offline renderer (chunked-render.mjs / offline-render-v2.mjs) runs a tight audio-processing loop that blocks the Node.js event loop. If you run it in your main OpenClaw session, it will kill the gateway after ~30 seconds (the heartbeat timeout).

✅ Correct: spawn a sub-agent or use background exec
❌ Wrong:   run the renderer inline in your main conversation

Always do this:

# Background exec with timeout
exec background:true timeout:120 command:"node src/runtime/chunked-render.mjs src/compositions/my-track.js output/my-track.wav 20"

Or spawn a sub-agent:

sessions_spawn task:"Render strudel-music composition: node src/runtime/chunked-render.mjs ..."

This is the #1 way to break things. Don't skip this.

Quick Start

# 1. Setup
cd ~/.openclaw/workspace/strudel-music
npm run setup              # installs deps + downloads samples (~11MB)

# 2. Verify
npm test                   # 12-point smoke test

# 3. Render
node src/runtime/chunked-render.mjs assets/compositions/fog-and-starlight.js output/fog.wav 16
ffmpeg -i output/fog.wav -codec:a libmp3lame -b:a 192k output/fog.mp3

Commands

Composition Workflow

1. Parse prompt → select mood, key, tempo, instruments (see references/mood-parameters.md)
2. Write a .js composition using Strudel pattern syntax
3. Render (in background!):

   node src/runtime/chunked-render.mjs <file> <output.wav> <cycles> [chunkSize]

4. Convert to MP3:

   ffmpeg -i output.wav -codec:a libmp3lame -b:a 192k output.mp3

5. Post the MP3 as attachment or stream to Discord VC

Discord VC Streaming

node src/runtime/offline-render-v2.mjs assets/compositions/combat-assault.js /tmp/track.wav 12 140
ffmpeg -i /tmp/track.wav -ar 48000 -ac 2 /tmp/track-48k.wav -y
node scripts/vc-play.mjs /tmp/track-48k.wav

WSL2 users: enable mirrored networking (networkingMode=mirrored in .wslconfig) or VC streaming will fail silently (NAT breaks Discord's UDP voice protocol).

Sample Management

Directory Layout

Samples live in samples/. Any directory of WAV files is auto-discovered.

samples/
├── strudel.json          ← sample map (pitch info, paths)
├── kick/
│   └── kick.wav
├── hat/
│   └── hat.wav
├── bass_Cs1/
│   └── bass_Cs1.wav      ← pitched sample (root: C#1)
├── synth_lead/
│   └── synth_lead.wav     ← pitched sample (root: C#3, declared in strudel.json)
└── bloom_kick/
    └── bloom_kick.wav     ← from audio deconstruction

strudel.json Format

Maps sample names to files with optional root note declarations. The renderer uses this as the authoritative source for pitch detection.

{
  "_base": "./",
  "kick": { "0": "kick/kick.wav" },
  "bass_Cs1": { "cs1": "bass_Cs1/bass_Cs1.wav" },
  "synth_lead": { "cs3": "synth_lead/synth_lead.wav" }
}

• Keys with note suffixes (_Cs1, _D2) declare the root pitch
• Unpitched samples use "0" as the key
• Always declare root notes for pitched samples — without it, the renderer defaults to C4, causing wrong transpositions (see docs/KNOWN-PITFALLS.md)

Managing Packs

bash scripts/samples-manage.sh list              # show installed packs
bash scripts/samples-manage.sh add <url>          # download from URL
bash scripts/samples-manage.sh add ~/my-samples/  # add local directory

Ships with dirt-samples (153 WAVs, CC-licensed). Security: downloads enforce size limits (STRUDEL_MAX_DOWNLOAD_MB, default 10GB), MIME validation, optional host allowlist (STRUDEL_ALLOWED_HOSTS).

Composition Guide

Pattern Basics

CC0 / Free packs (just download and drop in samples/):

• Dirt-Samples — 800+ samples (full pack, we ship a subset)
• Signature Sounds – Homemade Drum Kit (CC0) — 150+ one-shots
• Looping – Synth Pack 01 (CC0) — synth one-shots + loops
• artgamesound.com — CC0 searchable aggregator

Your own packs: Export from any DAW (Ableton, FL Studio, M8 tracker, etc.) as WAV directories. Strudel doesn't care where they came from — it's just WAV files in folders.

Named banks (Strudel built-in, requires CDN access):

sound("bd sd cp hh").bank("RolandTR909")
sound("bd sd hh oh").bank("LinnDrum")

WSL2 Note

If running on WSL2 and streaming to Discord VC, enable mirrored networking:

# %USERPROFILE%\.wslconfig
[wsl2]
networkingMode=mirrored

Then wsl --shutdown and relaunch. Without this, WSL2's NAT breaks Discord's UDP voice protocol — the bot joins the channel but no audio flows because IP discovery packets can't traverse the NAT return path. Mirrored mode eliminates the NAT by putting WSL2 directly on the host's network stack.

This only affects VC streaming. Offline rendering and file posting work in any networking mode.

Platform Requirements

Two tiers, depending on what you need:

Compose & Render (JS-only)

• Node.js 18+ (22+ recommended for stable OfflineAudioContext)
• ffmpeg (MP3/Opus conversion)
• Works everywhere — x86_64, ARM64, WSL2, bare metal, containers.
• No Python. No GPU. No ML stack.

Full Pipeline (audio deconstruction with Demucs)

Everything above, plus:

• Python 3.10+
• pip packages: demucs, librosa, numpy, scipy, scikit-learn, torch
• ~2GB disk for PyTorch + Demucs model weights (downloaded on first run)
• Optional: NVIDIA GPU + CUDA toolkit for ~5× Demucs speedup

Install the Python deps:

pip install demucs librosa numpy scipy scikit-learn torch

If Python deps are missing, composition and rendering still work — you just can't do stem extraction. The skill should fail gracefully with a message, not a stack trace.

Full Pipeline (Audio Deconstruction)

If you have an MP3 and want to extract instruments from it, build sample racks, and compose with the extracted material — that's the full pipeline. It goes:

MP3 → Demucs (stem separation) → librosa (analysis) → sample slicing → Strudel composition → render → MP3

This is a 4–8 minute process for a typical track. See docs/pipeline.md for the complete stage-by-stage breakdown with commands, timings, and resource requirements.

Quick version

# 1. Separate stems (Python/Demucs)
python -m demucs input.mp3 --out ./stems

# 2. Analyze + slice (see docs/pipeline.md for details)
# Currently semi-manual — analysis scripts in development

# 3. Write composition referencing sliced samples
# 4. Render
bash scripts/dispatch.sh render my-composition.js 16 120

# 5. Convert
ffmpeg -i output.wav -c:a libmp3lame -q:a 2 output.mp3 -y

Timings (ballpark)

Total (4-min track, CPU): 4–8 minutes. Compose + render only (no Demucs): 2–3 minutes.

⚠️ Session Safety — READ THIS

The full pipeline takes 4–8 minutes. Composition + render alone takes 2–3 minutes.

DO NOT run this inline in a Discord channel interaction or primary OpenClaw session. The 30-second response timeout will kill the process mid-render. There is no supervisor to recover. The skill will appear broken — silence, no output, no error message.

How to run safely

From an OpenClaw agent (correct):

sessions_spawn({
  task: "Render strudel composition: /strudel dark ambient tension, 65bpm",
  mode: "run",
  runTimeoutSeconds: 600  // 10 minutes — generous for full pipeline
})

Background process (also correct):

exec({ command: "bash scripts/dispatch.sh render ...", background: true })

Direct CLI (fine for testing):

bash scripts/dispatch.sh render assets/compositions/fog-and-starlight.js 16 72

What to tell the user: "Rendering takes a few minutes — I'll post the audio when it's ready." Don't leave them hanging with no feedback.

What NOT to do

// WRONG — will timeout after 30s in Discord context
exec({ command: "bash scripts/dispatch.sh render ..." })

// WRONG — blocking the main session for minutes
// (anything inline that takes >30s)

Learning Resources

Detailed documentation lives in docs/:

Start with composition-guide.md if you're writing patterns. The space-separated vs angle-bracket distinction is the #1 source of bugs (gain explosions, distortion, memory crashes). The guide covers it with real case studies.

How It Works

The offline renderer uses node-web-audio-api (Rust-based Web Audio for Node.js) for real audio synthesis:

1. Pattern evaluation — @strudel/core + @strudel/mini + @strudel/tonal parse pattern code into timed "haps"
2. Audio scheduling — Each hap becomes either:
   • An oscillator (sine/saw/square/triangle) with ADSR envelope, biquad filter, stereo pan
   • A sample (AudioBufferSourceNode) from the samples directory, with pitch shifting
3. Offline rendering — OfflineAudioContext.startRendering() produces complete audio
4. Output — 16-bit stereo WAV at 44.1kHz → ffmpeg → MP3/Opus

Note on mini notation: The renderer explicitly calls setStringParser(mini.mini) after import because Strudel's npm dist bundles duplicate the Pattern class across modules. Same class of bug as openclaw#22790.

Composition Reference

Tempo

setcpm(120/4)  // 120 BPM

stack(
  s("bd sd [bd bd] sd").gain(0.4),           // drums (samples)
  s("[hh hh] [hh oh]").gain(0.2),            // hats
  note("c3 eb3 g3 c4")                       // melody
    .s("sawtooth")
    .lpf(sine.range(400, 2000).slow(8))      // filter sweep
    .attack(0.01).decay(0.3).sustain(0.2)    // ADSR envelope
    .room(0.4).delay(0.2)                    // space
    .gain(0.3)
)