Diffusers

Diffusers

Use this skill when a task involves Hugging Face Diffusers APIs, pipeline wiring, model loading, scheduler configuration, adapters, training command planning, modular pipelines, checkpoint conversion, or maintaining the Diffusers repository.

Start Here

1. Check whether the user is using Diffusers as a package, editing a Diffusers checkout, or converting/training model assets.
2. Verify install and optional backends with scripts/check_diffusers_environment.py when imports, CUDA, CLI availability, or optional dependencies are uncertain.
3. Route to the narrowest sub-skill below instead of reading every reference.
4. Keep model downloads, Hub pushes, training runs, and conversion jobs opt-in; many Diffusers workflows are network-, credential-, GPU-, or memory-sensitive.
5. Read references/repo-provenance.md before deciding whether this skill is stale for a current Diffusers checkout.
6. Use references/troubleshooting.md for cross-cutting install/import/backend failures before diving into workflow-specific troubleshooting.

Installation Baseline

For normal package use, install Diffusers with the backend needed for the task:

python -m pip install diffusers torch accelerate transformers safetensors

For source-checkout development, install editable package dependencies in an isolated environment:

python -m pip install -e .
python -m pip install torch accelerate transformers safetensors

Add extras only when the selected workflow needs them:

• Training recipes commonly need accelerate, datasets, protobuf, tensorboard, Jinja2, peft, and timm.
• Quantization and accelerator paths may need bitsandbytes, gguf, optimum-quanto, torchao, nvidia-modelopt, xFormers, ONNX Runtime, OpenVINO, or vendor-specific packages.
• Flax/JAX, ONNX, TensorRT, Core ML, and other backend workflows require separate backend-specific installs.

Minimal import check:

python - <<'PY'
import diffusers
print(diffusers.__version__)
from diffusers import DiffusionPipeline, DDPMScheduler
print(DiffusionPipeline, DDPMScheduler)
PY

Route by Task

• Pipeline inference and serving: use sub-skills/pipelines-and-inference/SKILL.md for DiffusionPipeline.from_pretrained, AutoPipeline*, text-to-image, img2img, inpainting, ControlNet/T2I-Adapter/IP-Adapter execution context, callbacks, batching, seeds, device maps, offload, local/offline loading, and server-safe invocation.
• Schedulers and sampling: use sub-skills/schedulers/SKILL.md for DDIM/DDPM/Euler/DPM-Solver/FlowMatch/LCM schedulers, set_timesteps, custom timesteps/sigmas, prediction_type, Karras/AYS settings, scheduler config round-trips, and sampler troubleshooting.
• Adapters and loaders: use sub-skills/adapters-and-loaders/SKILL.md for LoRA/PEFT, textual inversion, IP-Adapter, T2I-Adapter, ControlNet loading, single-file checkpoints, adapter fusion/unloading, state-dict validation, and local-file loading plans.
• Training recipes: use sub-skills/training-recipes/SKILL.md for DreamBooth, LoRA, textual inversion, text-to-image, ControlNet, T2I-Adapter, InstructPix2Pix, SDXL, SD3, Flux, dataset layout checks, and accelerate launch planning.
• Modular pipelines: use sub-skills/modular-pipelines/SKILL.md for ModularPipeline, pipeline blocks, states, component managers, sequential/loop blocks, custom block packaging, and modular-pipeline tests.
• Conversion, CLI, and maintenance: use sub-skills/conversion-and-maintenance/SKILL.md for diffusers-cli, environment reports, safe conversion planning, ONNX/export notes, copied-code maintenance, dummy dependency checks, style, and focused repo tests.

Boundary Rules

• Do not start training, conversion, Hub upload, benchmark, or long inference jobs without explicit user confirmation.
• Do not assume a GPU-specific package is installed just because the host has GPUs; run the environment checker and inspect torch.cuda.is_available().
• Do not rely on original Diffusers repo docs, examples, or scripts when using this skill as a standalone runtime skill. The sub-skills include distilled references and safe helpers.
• When the user is editing a Diffusers checkout, follow the repo's copied-code policy: do not edit # Copied from ... blocks directly unless intentionally breaking the copy link; run copy/style checks before PR handoff.
• Keep pipeline execution guidance separate from training and conversion guidance. Loading an adapter for inference belongs to adapters/loaders plus pipelines; training that adapter belongs to training recipes.
• Treat original repo tests and examples as native verification candidates for a checkout, not as runtime dependencies for this skill.