Yue2 Music

YuE2 Music

Turn a musical request into a reproducible song and an audible comparison. Retain the original song and its plan before changing anything.

Choose the workflow

• Generate with editable melody and harmony: YuE2 cot="full" -> ABC -> song
• Generate with a melody plan and free accompaniment: YuE2 cot="melody" -> chord-free ABC -> song
• Generate without symbolic planning: YuE2 cot="off" -> song (no editable ABC)
• Cover a recording: SheetSage2 -> inspect and correct ABC -> strip chords -> YuE2 melody
• Cover an ABC melody: inspect or convert native ABC -> strip chords -> YuE2 melody
• Change harmony, instruments, tempo, structure or lyrics: copy the full plan -> edit ABC or text -> regenerate
• Agentic editing: export plan and baseline -> bounded editing agent -> check invariants -> render -> compare
• Analyse continuous musical features: MERT2, see models-and-setup.md

Set up

Read models-and-setup.md for model IDs, install pins and licences.

1. Work from a project directory, never inside this skill folder. <skill> in commands below is this skill's directory.
2. Create .venv-yue2 in the project and install the YuE2 runtime from the official GitHub repository source.
3. Create .venv-sheetsage2 separately (dependency pins differ) and download the SheetSage2 snapshot to models/SheetSage2.
4. Record every model revision you download.
5. Run on the supported baseline: one request at a time, BF16-capable NVIDIA GPU with 24 GB VRAM, default YuE2 settings. On Apple Silicon follow the MPS section of models-and-setup.md; scripts auto-detect the device.
6. Scripts default to Hub model IDs; see each script's --help for local snapshot and revision flags. Use a fresh output directory per run.

Generate and retain the plan

Copy assets/prompt.json to requests/<id>.json and edit it. Put genre, instruments, vocal character, language and tempo in style; put section tags and the actual words in lyrics. Keep implementation notes out of lyrics.

.venv-yue2/bin/python <skill>/scripts/run_yue2.py generate --request requests/song.json --output outputs/pop
.venv-yue2/bin/python <skill>/scripts/run_yue2.py all-modes --request requests/song.json --output outputs/modes
.venv-yue2/bin/python <skill>/scripts/run_yue2.py plan --request requests/song.json --output outputs/plan

Inspect result.json, truncation, score.abc, request.json and audio. The helper saves native artefacts including exact tokens and latent.npy; keep them. Preserve every requested mode and every failure. A playable file does not establish musical quality.

Read generation-and-covers.md for the Python interface, exact plan continuation, CFG, sampling and cached decoding. For a niche or extreme genre, or when a result sounds like the mainstream version of what was asked, apply its "Steer away from generic renditions" section before changing the prompt again.

Cover a recording

1. Transcribe in the SheetSage2 environment. Select the vocal melody or the full lead melody including instrumental passages.
2. Inspect warnings. Correct missed notes, meter or key before attributing errors to YuE2. Preserve source audio and raw transcription.
3. Export chord-free ABC. When dropping a part, select the retained voice explicitly; stripping chords alone should preserve both melodic voices and their rests.
4. Render with cot="melody", target style and suitable lyrics. To retain the original harmony as well, transcribe in full and use cot="full".

.venv-sheetsage2/bin/python <skill>/scripts/transcribe.py reference.wav --task melody-full --output outputs/transcription
python3 <skill>/scripts/abc_tools.py strip-chords outputs/transcription/score.abc outputs/cover.abc

# The request supplies target style and lyrics.
.venv-yue2/bin/python <skill>/scripts/run_yue2.py generate --request requests/cover.json --cot melody \
  --abc-file outputs/cover.abc --output outputs/cover-song

Edit or delegate an edit

Read editing-workflows.md and abc-editing.md before changing a score. Create a task per step below, each with its completion criterion, then work them to completion.

1. Render a baseline from the full plan. Freeze its directory.

2. Define invariants: exact pitches; pitch plus rhythm; contour only; or bounded melodic adaptation. Name the voices, passages, lyrics, instruments, tempo, meter and structure covered.

3. Delegate when possible: fill in the edit brief and give a score-editing agent the raw ABC, prompt, lyrics and requested change. Request a new ABC, revised style or lyrics as needed, and an edit manifest. Give a separate reviewer the before/after artefacts and constraints. Without delegation, perform each stage yourself. Keep model generation sequential per GPU.

4. Check musical events, not character strings: ties, accidentals and compressed rests matter.

   python3 <skill>/scripts/abc_tools.py inspect edits/jazz.abc
   python3 <skill>/scripts/abc_tools.py compare outputs/plan/score.abc edits/jazz.abc --voices Vocal
   .venv-yue2/bin/python <skill>/scripts/run_yue2.py generate --request edits/jazz.json --cot full \
     --abc-file edits/jazz.abc --output outputs/jazz

   Add --allow-tempo-change for intentional tempo changes. Exact comparison fails on intentional rhythm changes; audit the permitted differences from its report and label the result accordingly.

5. Regenerate after changing style, lyrics or ABC. Old acoustic latents can be decoded again but do not contain a musical or lyric edit.

6. Compare full songs and short passages around the edit. Revise when the requested effect fails. Retain each attempt with its actual prompt.

For lyric translation, follow the syllable mapping steps in abc-editing.md and keep the sidecar. Use ASR/PER and listening as separate evidence.