Simulation Rendering

Simulation and rendering

Use this sub-skill when the task concerns the MuJoCo model/data handles behind a MyoSuite environment, rendering pixels or depth, selecting cameras, preserving or restoring simulator state, or diagnosing a viewer/display failure. It covers the base MuJoCo backend in MyoSuite 2.x. It does not teach task selection, XML mutation or IK, MJX/JAX acceleration, or RL training.

Route first

• Choose environment workflows for registry lookup, task configuration, reset/step semantics, or environment-specific observation and reward behavior.
• Choose model editing and kinematics for XML changes, MjSpec editing, site/body edits as a modeling operation, or IK.
• Choose MJX acceleration for JAX, MJX, CUDA, batching, or accelerator performance.
• Choose training integration for policies, learners, checkpoints, or long-running experiments.

Fast decision: which rendering path?

1. Headless or CI: load mujoco.MjModel/mujoco.MjData, step with mujoco.mj_step, and use MJRenderer.render_offscreen(...), or run the bundled safe checker: python scripts/check_mujoco_xml.py --help.
2. A window is explicitly wanted: env.mj_render() delegates to env.mj_renderer.render_to_window(), which creates a native passive viewer on first use. This is display-dependent and is not verification ground truth.
3. Raw XML diagnosis: run python scripts/check_mujoco_xml.py --xml MODEL.xml --render none first. It loads and steps without creating a viewer. Add --render offscreen only when pixel output is required.

Never use myosuite.utils.examine_sim as a headless check: its documented workflow calls mujoco.viewer.launch(...) and is intentionally onscreen. Its safe native candidate is python -m myosuite.utils.examine_sim --help only.

Environment rendering contract

from myosuite.utils import gym

env = gym.make("myoElbowPose1D6MRandom-v0")
env.reset(seed=1234)
frame = env.mj_renderer.render_offscreen(
    width=320, height=240, camera_id=-1, rgb=True
)
env.close()

• env.mj_model is the compiled mujoco.MjModel; env.mj_data is its live mujoco.MjData. The base environment constructs env.mj_renderer as an MJRenderer for those handles.
• camera_id=-1 means MuJoCo's free camera. A named camera string or numeric camera id selects a model camera. Use a model camera name only after checking that it exists; a missing name is a model/rendering error, not a display fix.
• render_offscreen returns an RGB numpy array for the default call. The detailed return matrix for RGB/depth/segmentation is in rendering-api.md.
• env.mj_render() is deliberately different: it opens/synchronizes a window and returns no frame. Do not call it in a server, CI job, or headless synthetic case.
• env.viewer_setup(distance=..., azimuth=..., elevation=..., lookat=..., render_actuator=..., render_tendon=...) stores free-camera and visualization settings for the next window or offscreen scene. Camera/output details and the distance adjustment are documented in rendering-api.md.

Deterministic headless rollout

For a raw model, the safe contract is:

python scripts/check_mujoco_xml.py \
  --xml MODEL.xml --render offscreen --frames 8 \
  --width 320 --height 240 --output-dir ./mujoco-check

The command never calls mujoco.viewer or launch_passive. It prints model sizes, steps the requested number of frames, and writes one binary PPM image per rendered frame under the supplied output directory (for example, mujoco-check/frame-0000.ppm). --render none is the default and creates no output directory. The script accepts optional comma-separated --qpos and --ctrl; it validates their lengths before applying them. See the script's --help output for the complete input/output contract.

For a policy/environment rollout, use MujocoEnv.examine_policy_new only when its policy API is already available. Its supported render values are "onscreen", "offscreen", and "none"; offscreen mode accumulates RGB frames and writes an MP4 per episode through imageio. Set an explicit output_dir, filename, frame_size, and finite horizon. The safe raw-model script is preferred for a small, dependency-light rendering check.