Pie Testing

🧠 Brains complement: IF an unreal-engine-skills-manager tool (external MCP) exists in this session, call it with {action: "load", skill: "automation-and-testing"} for UE domain knowledge on this topic — correct APIs, architecture, best practices — and treat it as the rubric for any review / "best practices" question. If no such tool is available (e.g. running under Claude Code or Codex without that MCP), skip this line entirely and proceed with this skill alone — do NOT attempt the call.

Play-In-Editor (PIE) Testing Skill

PIE is the only way to validate runtime behavior — Blueprint logic, AI ticking, animation, input, widget interaction, gameplay events. Without starting PIE, your "fix" is unverified.

PIE control — engine EditorAppToolset

Generic PIE start/stop/status is owned by Unreal 5.8's native EditorToolset.EditorAppToolset, invoked through call_tool (run describe_toolset on it for exact action names/params):

call_tool(toolset="EditorToolset.EditorAppToolset", tool="IsPIERunning")
call_tool(toolset="EditorToolset.EditorAppToolset", tool="StartPIE")
call_tool(toolset="EditorToolset.EditorAppToolset", tool="StopPIE")

PIE start/stop/query is not on WidgetService anymore for general testing — use the engine toolset above. WidgetService still owns the widget-in-PIE validation helpers below.

Standard Test Loop

For repeatable multi-step verification, prefer WorkflowService.run_scenario() over hand-composing the loop. It queues a state machine on editor ticks, waits for actual PIE readiness, scopes log assertions to the scenario start, uses VibeUE's focus-free input injection, captures evidence, and always tears PIE down on pass, failure, or cancellation. Poll get_scenario(id) until its status is passed, failed, cancelled, smoke_passed, or stale:

import json, unreal
spec = {
  "name": "secondary fire consumes ammo",
  "preflight": {"save_dirty_assets": True, "compile_blueprints": ["/Game/Weapons/BP_Rifle"]},
  "steps": [
    {"action":"start_pie"},
    {"action":"wait_for_pie", "timeout_seconds":30},
    {"action":"inject_action", "path":"/Game/Input/IA_Fire_Secondary"},
    {"action":"wait", "seconds":0.25},
    {"action":"assert_log", "contains":"SecondaryFire"},
    {"action":"capture_game", "name":"after-secondary-fire"}
  ],
  "teardown": {"stop_pie": True}
}
queued = json.loads(unreal.WorkflowService.run_scenario(json.dumps(spec)))
# Poll in separate tool calls; do not block the editor game thread with time.sleep.

Use the lower-level primitives below for interactive investigation, one-off probes, or actions not in the scenario schema. Never spin/sleep inside one editor Python call while waiting for a scenario.

Assertions and evidence freshness

A normal scenario must declare at least one assert_log, python_assert, or python_assert_number step. Completing input/wait/capture steps alone does not verify gameplay. For a deliberate boot-only check, set "smoke": true: an assertion-free run finishes with status="smoke_passed" and passed=false. Pollers must treat smoke_passed and stale as terminal alongside passed, failed, and cancelled.

python_assert compares the Python result with a required string expected. python_assert_number evaluates an expression and requires a finite numeric result:

{"action":"python_assert_number", "expression":"0.1 + 0.2",
 "operator":"eq", "expected":0.3, "tolerance":0.00001}

Operators are eq, lt, le, gt, and ge; nonnegative tolerance is supported only for eq and defaults to zero. For gameplay, use an expression reading a live actor/property instead of the arithmetic example. Expected/actual values are recorded per assertion; an expression error, nonnumeric result, or unmet comparison fails the scenario. Do not cache PIE objects in globals. If an assert_log supplies both contains and not_contains, both conditions must hold. Log assertions use the active log (including -abslog overrides); unavailable or truncated logs fail the assertion, including negative checks. Reports include assertionsDeclared and assertionsEvaluated, including failed assertions.

To detect stale evidence, add a top-level dependencies array of actual file paths relative to the project directory (or absolute paths), for example Content/Ships/BP_Ship.uasset and relevant source/config files. Dependencies must exist. VibeUE hashes their contents after preflight, records the scenario hash and engine version, and automatically tracks its plugin DLL when dependencies are provided. It does not persist the submitted Python source as provenance.

get_scenario() checks those hashes again for completed reports, including reports loaded from disk. A changed/missing file or changed engine version produces validity="stale"; a historical pass then returns status="stale", passed=false, and historicalPassed=true. verifiedCurrent=true requires passing assertions and unchanged tracked inputs. Without dependencies, validity="untracked" and verifiedCurrent=false, even if assertions passed. Freshness covers only explicitly listed files plus the plugin DLL: it does not infer transitive asset dependencies, detect unsaved edits, or prove a newly changed project binary is loaded. Save and compile first, list all relevant inputs, and rerun after changes. Hashes are freshness checks, not tamper-proof attestations. Large dependency sets increase polling cost.

These verification improvements were inspired by PageMastr/Gatekeeper, particularly its assertion coverage and source-bound verdicts. The VibeUE implementation is independently written for Unreal.

# 1. Make sure you're starting from a clean state
call_tool(toolset="EditorToolset.EditorAppToolset", tool="StopPIE")   # no-op if not running

# 2. Start the session (uses the editor's current PIE settings — default map, viewport)
call_tool(toolset="EditorToolset.EditorAppToolset", tool="StartPIE")

# 3. Let the test run / inspect log output (LogsToolset) / interact via other services
# 4. Stop when done
call_tool(toolset="EditorToolset.EditorAppToolset", tool="StopPIE")

Net mode from Python — EngineSettingsService

StartPIE uses the editor's saved PIE settings, so set the net mode BEFORE starting the session. EngineSettingsService reads and writes ULevelEditorPlaySettings directly and persists the change, so a dedicated-server PIE gate no longer means quitting the editor to hand-edit EditorPerProjectUserSettings.ini and relaunching:

import unreal

# Read the current PIE multiplayer settings
info = unreal.EngineSettingsService.get_pie_settings()
print(info.net_mode, info.num_clients, info.run_under_one_process)

# Set dedicated-server PIE: 1 client, all windows in one process
ok = unreal.EngineSettingsService.set_pie_settings("Client", 1, True)   # returns True only on verified write
# Verify by readback, never by return value alone
assert unreal.EngineSettingsService.get_pie_settings().net_mode == "Client"

• net_mode maps EPlayNetMode: "Standalone" | "ListenServer" | "Client". "Client" is dedicated-server PIE — the editor's "Play As Client", where a windowless dedicated server is spawned behind the scenes and PIE instance 0 is that server ("DedicatedServer" is accepted as an alias for "Client").
• set_pie_settings refuses an unknown net mode or a num_clients outside 1-10 (returns False, logs a Warning), and returns False if any written value fails to read back.
• These are per-USER config, not project config — they live in Saved/Config/<Platform>/EditorPerProjectUserSettings.ini and follow the machine, not the repo. Do not expect a teammate or CI to inherit them; set them from the gate itself.

Validating widgets in PIE — WidgetService

VibeUE keeps a small set of widget-in-PIE helpers on unreal.WidgetService (run them via execute_python_code). These spawn and inspect live widget instances once PIE is running:

import unreal

# After StartPIE, spawn a widget instance into the running viewport
handle = unreal.WidgetService.spawn_widget_in_pie("/Game/UI/WBP_HUD", 0)

# Read a live property off the running instance
val = unreal.WidgetService.get_live_property(handle, "HealthText", "Text")

# Tear it down before stopping PIE
unreal.WidgetService.remove_widget_from_pie(handle)

unreal.WidgetService.is_pie_running() also still exists and is handy from inside Python; for tool-level control prefer the engine EditorAppToolset actions above.

Spawning test actors in PIE — PIEActorService

Why this exists: you cannot spawn an actor into a running PIE world from Python any other way. unreal.World / unreal.GameplayStatics expose no spawn (the deferred-spawn entry points are BlueprintInternalUseOnly), EditorActorSubsystem.spawn_actor_from_class targets the editor world, and the engine SceneTools toolset refuses with "Cannot create actors while PIE is active". PIEActorService resolves the live PIE world by role and spawns a transient actor into it — a blocker, a test dummy, a trigger volume — with no Blueprint and no editor-world contamination. Run it via execute_python_code.

Selector grammar (strict — anything else returns INVALID_SELECTOR):

Worlds come from the PIE world contexts only; the editor world and the ~100 stale /Memory/UEDPIE_* shells are never used. No PIE at all → PIE_NOT_RUNNING.

Authority rule: a spawn into a client world is cosmetic and never replicates, so it is refused with CLIENT_REQUIRES_OPT_IN unless you pass allow_client_local=True. Spawn on "server" for anything that must exist for every player.

import unreal, json

# Drop a static-mesh actor as a blocker in front of the host, then verify and clean up.
t = unreal.Transform(location=unreal.Vector(500, 0, 100))
res = json.loads(unreal.PIEActorService.spawn_actor(
    "server", "/Script/Engine.StaticMeshActor", t))
assert res["success"], res
handle = res["handle"]          # "<session_serial>:<guid>" — dies with this PIE session
print(res["actor_path"], res["net_mode"], res["is_authority"])

# ... run whatever movement / collision check needs the blocker ...

# Idempotent teardown before StopPIE.
print(unreal.PIEActorService.destroy_all())     # {"success": true, "destroyed": 1, "already_gone": 0}

resolve_world("server") returns {success, world_path, net_mode, pie_instance, is_authority, actor_count} and is the cheap way to confirm a session's role before spawning. destroy_actor(handle) is idempotent per handle: it destroys the actor and keeps the record, so calling it again returns success: true, already_gone: true (never UNKNOWN_HANDLE). A consumed handle stays in list_spawned() as alive: false until PIE ends — list_spawned() is the full spawn ledger for the session, not just the live actors — and destroy_all() counts an already-consumed actor under already_gone rather than destroyed.

Error codes: INVALID_SELECTOR, PIE_NOT_RUNNING, WORLD_NOT_FOUND, AMBIGUOUS_WORLD, CLASS_NOT_FOUND, NOT_AN_ACTOR_CLASS, CLASS_ABSTRACT, CLASS_DEPRECATED, CLASS_REINSTANCED, INVALID_TRANSFORM, INVALID_COLLISION_HANDLING, CLIENT_REQUIRES_OPT_IN, SPAWN_REJECTED (the placement collided under the chosen collision policy), UNKNOWN_HANDLE, STALE_HANDLE.

Gotcha — handles die with the PIE session. EndPIE bumps an internal session serial and clears the registry, so a handle kept across a stop/start reports STALE_HANDLE. And the same Python-globals rule as everywhere else applies with teeth here: release any global holding a spawned actor before StopPIE (del blocker / set to None, or vibeue.release_globals), or the editor asserts on PIE teardown ("Object from PIE level still referenced"). The returned JSON is a string, so keeping res/handle is safe — only a live actor reference is the hazard.

Driving gameplay input in PIE — InputService (issue #550)

Never remap game input assets or send OS keystrokes (SendKeys/AppActivate) to test a mechanic — inject input directly; no OS window focus needed:

import unreal

# Fire an Enhanced Input action once: QUEUED for the next input tick, released on the tick after.
# X/Y/Z map onto the action's value type; Boolean uses X != 0.
print(unreal.InputService.inject_action("/Game/Input/IA_Fire_Secondary"))

# Hold it for 1.5 s of real time, then release (returns at once; stop_injection releases early).
print(unreal.InputService.inject_action_for("/Game/Input/IA_Fire_Secondary", 1.5))

# Or send a raw key to the game viewport through Slate ("tap" = down+up; also "down"/"up",
# and "hold" with a duration: inject_key("RightMouseButton", "hold", 1.5)).
print(unreal.InputService.inject_key("SpaceBar"))

A Python call blocks the game thread, so PIE does not tick while it runs: inject in one call and read the game state in a LATER call, two or more frames on. While a hold is active the editor stays off its background frame rate, so an unfocused editor does not starve the hold. The action calls take pie_instance to pick a PIE world: -1, the default, is the first PIE world with a local player (the lowest instance number — the listen server's window, or client 1 under a dedicated server), and replies report the instance actually used. A hold is one hold per action and PIE world whatever the path spelling; holding a held key again extends it; StopPIE drops every hold, so nothing from an ended session is released into the editor or reported active in the next one.

All return JSON with success and an error_code naming the problem (PIE not running, NO_LOCAL_PLAYER / NO_PLAYER_CONTROLLER while PIE or a client's join is still starting — retry on a later call —, unknown key, ...). inject_key additionally rejects a Simulate In Editor session with SIMULATE_NOT_PLAY: Simulate has no player game viewport, so the Slate key events would be dropped. Use StartPIE (Play), not Simulate, when driving keys. Its handled_down / handled_up fields report whether a Slate widget consumed the event, not whether the key reached the player — a key the game polls with WasInputKeyJustPressed lands with handled_down: false. Verify from game state, not from those flags.