Debug UI Interactions

Debug native UI interactions

Read first

Read skills/test-via-api/SKILL.md for the production-runtime lifecycle and docs/architecture/rhai-integration.md for the typed input boundary. Use skills/coordinate-frames/SKILL.md when a screen hit, world position, camera frame, terrain point, or BigSpace grid is part of the failure.

Use the real application input path

Build or resolve the production binary as LUNCOSIM_BIN, launch exactly one windowed luncosim process with an explicit free API port, and wait for /api/ready before injecting input. The InjectWindowInput command and prelude/input.rhai helpers enqueue typed Bevy window events and exercise the application event path after event creation. They bypass the OS device, compositor, and winit event-delivery path, so they cannot verify that a physical mouse reaches the app. egui, HUI, picking, focus, input bindings, and authored Rhai tools process the injected Bevy events. Injected events do not mutate the native window cursor. Bevy Picking's PointerLocation is the shared application cursor state; cursor-driven consumers such as the transform gizmo read it through PrimaryMousePointer in lunco-interaction-core.

For a modifier gesture, use separate event phases and allow a frame between them when the result matters:

input_key_press("AltLeft");
input_pointer_move(x, y);
input_pointer_press("primary", x, y);
input_pointer_release("primary", x, y);
input_key_release("AltLeft");

Use KeyF for the default action binding only after checking the active input_bindings setting; authored tests should use the semantic binding or input_binding(...) rather than assuming a physical key. Use a secondary pointer event for context menus. Do not combine press and release events in one same-frame script step when testing focus, modifier state, or picking.

A route-point secondary click opens its authored context menu without changing scene selection. Selecting a point is separate from the Move route point action: select enables the generic gizmo, while Move selects its explicit point and arms click-to-place with a disposable ghost. That selected point remains the move target until placement or cancellation. Hover alone must not arm movement. The context gesture itself must not enable the transform gizmo. Ordinary clicks carry active_pointer_move { interaction_id, tool, hook, context } only for an interaction registered in their document. Placement consumes that context; an idle click must not discover armed tools or routes by traversing USD. The hover dispatcher scopes movement from the hit prim's document and carries the same-document selected/control paths as route context. The hit prim's registered LunCoPointerInteractionAPI must authorize that button as context; the generic viewport adapter applies its per-button blocking behavior before the ordered-hit pass, and route policy uses canonical hit paths rather than screen proximity. Picking can target a child collider or terrain LOD entity without UsdPrimPath; resolve the nearest ancestor prim before determining its document, as click routing does. The popup host registers the foreground menu rectangle with ScenePickGate as chrome, even when that rectangle lies inside the 3D viewport; menu clicks must not also start a gizmo drag. A semantic chord alone does not create a menu. Unarmed route-edit and selection clicks go through one scene_interaction Rhai policy; simulation possession accepts only an exclusive selection.replace intent. Spawn, terrain, attachment, and camera consumers are not yet under one captured gesture manager. The editor gizmo has a local captured lifecycle: a same-frame handle hit owns primary input through release or cancellation and suppresses preview pan for that gesture. A route fixture passing does not prove global viewport arbitration.

An authored pass-through hit can still emit its own Bevy pointer event. The shared scene dispatcher must stop that hit's ancestor propagation before de-duplicating the gesture, so a lower blocking or context target can receive it. Disposable scene previews that overlap interactive geometry must author pass-through behavior for the buttons they should not consume.

The retained Bevy UI backend shares the window target with scene picking. The Workbench's UI camera sorts above the scene, so unmarked Bevy UI nodes would block scene hits even when they are decorative. RuntimeUiPlugin requires UI pick markers and marks only visible authored press controls and draggable surface content as Pickable; keep that explicit target policy when adding UI nodes. Native pointer routing and tool dispatch run in the application input schedule before fixed simulation, so route context handling must not wait for a physics tick. Scene-pointer observers use the picked hit and viewport-aware chrome capture as the ownership decision. EguiFocus is published after picking and can still describe the old cursor location during the first event that leaves a menu; do not use that snapshot to reject a valid scene hit.

RunRhaiTool and RunRhaiToolHook callbacks use a bounded UI queue drained after picking in PreUpdate, before fixed simulation. Keep authored pointer and menu policy in those tool hooks rather than sending it through the general REPL queue or a fixed-tick scenario. Heavy synchronous work in an input hook would still occupy the application thread, so keep the hook bounded and move preparation or I/O to its owning asynchronous boundary. When a UI hook reads a collection of USD prims, request the member list once and use QueryUsdPrims for their needed attributes, schemas, or relationships; declare that provider in the caller's query_reads. Avoid per-member QueryUsdPrim loops in input policy because every read occupies the same application thread. Resolve a live USD entity with its owning document identity when several documents can mount the same authored path. Generic MoveEntity persistence resolves the write document from the moved entity's stage, not the active editor tab.

Scene Pointer<Move> and Pointer<Enter> observers may collect raw hits, but the viewport must dispatch movement only for a document with an active typed SetScenePointerMoveHook subscription. Supply a caller-owned interaction_id to both Set and Clear so stale cleanup cannot end a newer interaction. Coalesce the newest raw hit per pointer and picking frame. With no active subscriptions, drop samples before resolving scene identity; with subscriptions on other documents, perform the lightweight document lookup and drop the hit before coordinate conversion or terrain work. For an active interaction, a fallback terrain raycast runs at most once per pointer per picking frame; direct analytic surface hits do not need that fallback. Queue one typed hook after resolution. Clear the subscription when the interaction ends; document close, Twin close, and scene teardown own cleanup. Enter provides the first sample after a scene hit replaces a popup's previous-frame capture hit. Bevy can emit move events for both the pass-through preview and lower hits. Before deduplication, skip the preview event and stop its ancestor propagation; otherwise the preview can consume the cursor sample intended for terrain beneath it. Keep the hook presentation-only: update the live transform of the projected @view@ preview through the generic typed preview-transform command. It validates document and view-layer ownership and uses the canonical active-frame/parent-local conversion; hover must not edit the USD document, trigger projection, or sample a full terrain path. Route add and delete hooks send the accepted route-point snapshot to UpdateUsdCurveView; the presentation owner coalesces it and prepares sparse terrain-local strokes. The terrain shader paints them on its own fragments without changing USD generation. For a DEM route, require InspectUsdCurveView.projection = "terrain_surface", a positive surface_binding_count, and exactly one segment per authored leg; the separate mesh stays hidden. Right-click identity comes from a foreground terrain hit against the published stroke. Use route_surface_annotation.rhai through RunScenarioAsset for publication, long-path and missing-coverage evidence. Its positive fixture needs terrain covering a 40 m circle around the first waypoint and a connector 300 m away along positive X/Z. Pass its explicit view_owner scenario so the gate can isolate and restore that writer. The next primary click commits a moved route point through the canonical @runtime@ USD edit path, whose projected change updates the ribbon once. The route_interaction production gate verifies that Move selects and retains the target, the live ghost follows a coalesced injected terrain cursor trace, and document generation stays unchanged during preview. It sequences typed Bevy press/release events on separate task ticks and waits for observable state with the Rhai behavior tree. It does not verify OS, compositor, or winit input delivery; actual headful input remains a separate check. Do not use simulation-time sleeps or encode progression as numeric phase state. The repeatable production gate is assets/scenes/tests/editor/route_interaction/route_interaction.usda, run by scripts/run_editor_scene_tests.sh; it uses InjectWindowInput to send typed Bevy window events through picking and verifies the mounted fixture, waypoint hit, semantic context intent, unchanged pre-menu selection, live scene selection in the focused editor owner, explicit Move menu action, ghost placement, and deletion. This gate does not exercise physical OS mouse input. Bevy emits one click observer per entity in its previous hover map, whose iteration order is unspecified. The runtime scene router resolves from that same event-source map by hit depth and the authored policy for the actual button before it dispatches one scene event. The runner waits for /api/ready and requires the API Exit command and port release after every verdict.

Runtime-authored route edits belong in Twin @runtime@. Run scripts/run_scene_tests.sh --exact route_runtime_persistence to exercise a manifest-backed Twin through two production API sessions: add a point, verify the .lunco/runtime sidecar write, then reopen and require that the point is present before the scene's initial projection. This test is separate from the isolated editor fixture gate because the latter intentionally disables runtime-overlay I/O.

Coordinates are logical primary-window pixels. Obtain them from a current screenshot and record the window geometry used for the run. For an Editor USD preview, read InspectUsdViewport's measured image_rect and scale_factor to derive coordinates from the exact image area; keep authored fixture geometry and the screenshot in the test review. A coordinate is test input, not domain state: never use it to infer a USD position or replace the canonical BigSpace/frame conversion.