Gui Integration Test

Warp Integration Tests

Scope — GUI desktop app only. This skill applies to Warp's GUI desktop front-end (the app/ crate on the WarpUI pixel/GPU framework). It does not apply to the headless TUI front-end (crates/warp_tui; cell-grid TuiElement library under crates/warpui_core/src/elements/tui), which has its own components, tests, and change-verification workflow. For TUI work, see the tui-ui-guidelines, tui-testing, and tui-verify-change skills instead.

Use this skill for Rust integration tests in Warp's custom framework under crates/integration/.

These are not ordinary unit tests. They boot a real Warp app instance, give it an isolated test home directory, drive it with synthetic UI and terminal events, and poll assertions until success or timeout.

When an integration test is the right tool

Integration tests are the most expensive tests in the repo. They boot the app, they are orders of magnitude slower than a unit test, and they are the most likely to go flaky. Write one when the risk you are covering genuinely lives between components:

• Real terminal/PTY and shell-integration behavior, especially across bash, zsh, and fish.
• Wiring and configuration: settings, keybindings, and user preferences actually taking effect end to end.
• Cross-component flows a unit test cannot see — command palette into editor into terminal, focus and pane management, tab/window lifecycle.
• Input and rendering paths that only exist once a real app (and sometimes a real display) is running.
• Regressions for user-visible bugs that escaped unit tests.

Do not reach for an integration test when:

• The logic is deterministic and reachable in-process. Push it into a unit test (rust-unit-tests); it will run in milliseconds and point straight at the failure.
• You are re-covering branch logic a unit test already covers. This harness is for the seams between components, not for re-testing conditionals at full app-boot cost.
• You only need TUI rendering. That is tui-testing — this harness does not drive the TUI at all.
• What you actually want is a screenshot or a manual look. Use computer_use or the gui-integration-test-video skill instead of attaching weak assertions to a full app boot.

If a behavior is hard to reach from a unit test only because of how the code is structured, prefer fixing the structure over writing a slow test around it.

Framework map

The core pieces are:

• crates/integration/src/bin/integration.rs
  • Manual integration test runner binary.
  • Registers test names to Builder factories.
  • Runs exactly one named test per invocation.
• crates/integration/tests/common/mod.rs
  • The outer Rust test harness used by cargo test and cargo nextest.
  • Shells out to the integration binary.
  • Forwards a limited set of env vars (PATH, RUST_*, WARP_*, WARPUI_*, WGPU_*, display-related vars).
  • Re-runs tests up to 10 times when the integration binary exits with the special rerun code.
• crates/integration/src/test.rs
  • Module hub for integration tests.
  • Add new test modules here and pub use their functions so the runner can see them.
• crates/integration/tests/integration/ui_tests.rs
  • List of UI-oriented integration tests that nextest should run.
• crates/integration/tests/integration/shell_integration_tests.rs
  • List of tests that must run against every shell or a specific shell matrix.
• crates/integration/src/builder.rs
  • Warp-specific wrapper around the lower-level WarpUI integration builder.
  • Sets default timeout, hermetic home directory, shell rc files, user prefs, and real-display mode when requested.
• crates/warpui_core/src/integration/driver.rs
  • Executes steps, handles retries, precondition reruns, screenshots, video capture, artifact export, and on_finish.
• crates/warpui_core/src/integration/step.rs
  • Defines TestStep, input/event APIs, assertion polling, step-to-step data passing, and screenshot/recording hooks.
• app/src/integration_testing/
  • High-level helpers and assertions for common Warp behaviors.
  • Prefer these helpers over raw low-level event plumbing whenever they fit.

How the framework actually runs a test

1. A Rust test from crates/integration/tests/integration/*.rs calls run_integration_test("test_name").
2. That harness launches the integration binary with the test name.
3. The binary in crates/integration/src/bin/integration.rs looks up the name in register_tests(), builds the Builder, and turns it into a TestDriver.
4. Builder::build(...) creates an isolated temp directory, points HOME at it, writes minimal rc files, and initializes file-backed user preferences.
5. The driver runs each TestStep in order:
   • setup callbacks
   • synthetic events
   • actions
   • assertion polling until success or timeout
6. If an assertion returns PreconditionFailed, the binary exits with the rerun code and the outer harness retries the whole test.
7. On success, failure, or cancellation, the driver can run on_finish and export artifacts/runtime tags.

This means integration tests should be written for a hermetic environment. Do not rely on the developer's real shell dotfiles, home directory contents, or persisted Warp settings.

Where to put a new test

Add the actual test function in a module under crates/integration/src/test/.

Use these heuristics:

• Put the test in an existing module when it matches that feature area.
• Create a new module when the feature does not fit an existing one cleanly.
• Add the test to crates/integration/tests/integration/ui_tests.rs if it is primarily a UI/app behavior test.
• Add the test to crates/integration/tests/integration/shell_integration_tests.rs if it needs to run against every shell, or depends on a specific shell/set of shells.

Being present in crates/integration/src/test/*.rs is not enough. For a test to run under cargo nextest, it also needs to be listed in one of the macro files in crates/integration/tests/integration/.

Authoring checklist for a new test

When adding a new integration test, do all of the following:

1. Implement pub fn test_name() -> Builder in a module under crates/integration/src/test/.
2. Add the module to crates/integration/src/test.rs.
3. pub use the new module's exports from crates/integration/src/test.rs.
4. Add register_test!(test_name); in crates/integration/src/bin/integration.rs.
5. Add test_name to either:
   • crates/integration/tests/integration/ui_tests.rs, or
   • crates/integration/tests/integration/shell_integration_tests.rs
6. Default to making the test run in CI once it is added to one of those macro lists. Only mark it #[ignore] when the task explicitly calls for manual-only coverage or there is a concrete, documented reason it cannot run reliably in CI.
7. Run the test manually first, then through nextest once it is stable enough for the suite you chose.

Writing the test body

The normal shape is:

use crate::Builder;
use warp::integration_testing::step::new_step_with_default_assertions;
use warp::integration_testing::terminal::{
    clear_blocklist_to_remove_bootstrapped_blocks,
    execute_command_for_single_terminal_in_tab,
    wait_until_bootstrapped_single_pane_for_tab,
    util::ExpectedExitStatus,
};

pub fn test_example() -> Builder {
    Builder::new()
        .with_step(wait_until_bootstrapped_single_pane_for_tab(0))
        .with_step(clear_blocklist_to_remove_bootstrapped_blocks())
        .with_step(execute_command_for_single_terminal_in_tab(
            0,
            "echo hello".to_string(),
            ExpectedExitStatus::Success,
            "hello".to_string(),
        ))
        .with_step(
            new_step_with_default_assertions("Assert some UI state")
                .add_named_assertion("specific assertion name", |app, window_id| {
                    // inspect app state and return AssertionOutcome
                    warpui::integration::AssertionOutcome::Success
                }),
        )
}

Prefer a small number of focused steps with descriptive names over a huge monolithic test.

Builder guidance

Builder::new()

Start here almost every time.

Warp's wrapper automatically gives you:

• a per-test root directory
• isolated HOME
• generated rc files for Bash, Zsh, and Fish
• file-backed user preferences
• a default 2-minute hard timeout
• real-display support if WARPUI_USE_REAL_DISPLAY_IN_INTEGRATION_TESTS is present

with_setup(...)

Use this for filesystem or environment setup before the app runs.

Common patterns:

• utils.set_env("NAME", Some(value))
• creating files under utils.test_dir()
• writing fixture config files

Prefer this over reaching into the real filesystem.

with_user_defaults(...)

Use this to set persisted Warp preferences before the test starts.

This is the right tool for settings backed by user preferences rather than environment variables.

set_should_run_test(...)

Use this to gate tests on shell/platform/runtime capabilities when the test genuinely cannot run everywhere.

with_on_finish(...)

Use this for final verification or artifact inspection that should happen after all steps complete, such as checking that screenshots or recordings were written.

with_real_display()

Use this explicitly when the test needs a real display for frame capture or visual workflows. Video/screenshot tests should normally be manual or ignored in CI unless there is a stable real-display path.

TestStep guidance

TestStep is the unit of execution. Each step can have:

• setup callbacks
• input events
• actions
• assertions
• a timeout
• retry count
• failure handling

Start from helper constructors

Prefer:

• wait_until_bootstrapped_single_pane_for_tab(0)
• new_step_with_default_assertions("...")
• new_step_with_default_assertions_for_pane("...", tab, pane)

The default step helpers already assert:

• no pending model events
• no block executing

These are good baseline invariants for most UI interactions.

Prefer helper APIs over raw event plumbing

Use high-level helpers from app/src/integration_testing/ whenever possible:

• terminal command execution helpers
• block list helpers
• command palette helpers
• navigation helpers
• settings helpers
• workflow/file tree/notebook helpers

Drop to raw with_event(...), with_event_fn(...), or saved-position mouse events only when there is no suitable helper.

Use named assertions

Prefer add_named_assertion(...) over unnamed assertions. Named assertions make failure output and runtime tags much easier to interpret.

Use polling assertions instead of sleeps

Assertions are polled until success or timeout. Lean on that model instead of hardcoding sleeps.

Good pattern:

• trigger an event or action
• assert on the eventual UI/model state

Avoid brittle timing assumptions.

Use step data when one step computes something for the next

If a later step needs data from an earlier one, use:

• add_named_assertion_with_data_from_prior_step(...)
• StepDataMap

This is useful for saving measured positions, counts, IDs, or other values from prior frames.

Use retries sparingly

set_retries(...) can help for a legitimately retryable step, but do not use it to hide deterministic failures. Prefer making the step more robust first.

Use PreconditionFailed for genuinely environmental flakes

If the environment reaches a state where the rest of the test is invalid, return AssertionOutcome::PreconditionFailed(...) instead of failing hard. The outer harness can rerun the entire test up to 10 times. The existing bootstrap helper is a good model for this.

Use this deliberately. The rerun mechanism exists for conditions the test genuinely cannot control, such as bootstrap racing or shell startup timing. It is not a way to turn an intermittently failing test green. A real bug that reproduces one run in five will pass under rerun and ship to users.

Before reaching for PreconditionFailed, confirm the failure is actually environmental by looking at the failure rate and the failure mode:

for i in {0..50}; do
  RUST_BACKTRACE=full cargo run -p integration --bin integration -- test_name || break
done

If the same assertion fails in different ways, or fails while the environment looks fine, it is a product or test bug. Fix it rather than absorbing it into a rerun.

Designing a good integration test