Qv SDK E2E Create

SDK e2e Test Creation

Plan and scaffold e2e tests in packages/sdk/e2e for a new or changed SDK feature exposed through the public API.

When to use this skill

Applies to SDK changes in packages/sdk/ that touch the public API surface.

Use when:

• Adding a new public SDK function, model type, or capability.
• Changing an existing public SDK API in a way that affects runtime behaviour.
• User invokes /qv-sdk-e2e-create.
• User asks to "add e2e tests for <feature>" or similar.

Do NOT use for:

• Internal refactors that don't change the public surface.
• Unit tests inside packages/sdk/ (this skill covers only the e2e suite under packages/sdk/e2e).

Approach

Investigate first, then propose a concrete plan. Only ask the user for information that cannot be recovered from code or context.

1. Read the feature. Identify the new/changed exports, inputs, return type, model dependencies, and any existing examples under packages/sdk/examples/ or tests under packages/sdk/e2e/tests/.
2. Find comparable tests. Look for an analogous existing feature in tests/test-definitions.ts and its executor. Mirror its style unless there's reason to deviate.
3. Determine model-output testability (see §"Model-output strategy"). Propose a specific validator and a specific prompt/input that makes the output deterministic enough to assert.
4. Draft happy / sad / error cases as a concrete test-definition sketch.
5. Decide executor placement, consumer registration, and mobile constraints (see §"Placement and consumer constraints").
6. Select at most one smoke candidate (see §"Smoke policy").
7. Present the plan to the user. Include: feature summary, chosen validators with rationale, test definitions sketch, placement decision, desktop/mobile/Electron registrations, platform concerns, smoke pick. Ask clarifying questions only where genuine ambiguity remains (e.g. expected model behaviour on an edge case, preferred tolerance for a numeric-range).
8. After approval, scaffold the files and prompt the user to run locally with the exact commands for every consumer. Always run run:local:electron --filter <feature>- to verify either the handler or an intentional skip for Electron/Snap.

Model-output strategy

For any feature that invokes a model, do not default to shape-only checks. type validation proves nothing about model correctness and must be a last resort.

Pick the strongest achievable strategy:

1. Exact-ish output — constrain the prompt + deterministic params (temperature: 0, fixed seed, top_k: 1) so a known token must appear. Assert with contains-all. Example: prompt "Reply with only the word APPLE." → assert result contains APPLE.
2. Closed-set — enum-style prompt (known set of valid answers) → contains-any.
3. Numeric range — a score, similarity, duration, or embedding magnitude with known bounds → numeric-range. Pick bounds tolerant to minor model drift.
4. Regex structure — structured output (JSON keys, date format, language tag) → regex. Keep the pattern anchored and stable.
5. Shape-only fallback — type with minLength. Flag as weak coverage in the plan.
6. Error path — throws-error with a substring that is stable across SDK versions.
7. Custom function — use for deterministic but non-trivial checks like cosine similarity against a reference vector.

If the model's output is inherently non-deterministic and cannot be constrained, say so in the plan and justify why shape-only or range-based coverage is the best achievable — do not silently ship a weak assertion.

Happy / sad / error minimum

Every public-API feature MUST have at minimum:

• Happy path — valid input, canonical output, strongest assertion achievable.
• Sad path — boundary or edge case that must still succeed (empty input, minimum ctx, longest accepted input, unusual but valid locale, streaming vs non-streaming).
• Error path — invalid/malformed input, missing asset, or exceeded constraint. Must throw with a matchable message via throws-error.

More cases are encouraged for multi-branch features.

Placement and consumer constraints

Executor placement (from packages/sdk/e2e/AGENTS.md):

• Pure SDK API, no Node stdlib, no RN APIs → tests/shared/executors/.
• Needs node:fs, node:path, process.cwd(), or other Node-only APIs → tests/desktop/executors/.
• Needs RN Platform, bundled assets, or anything specific to React Native → tests/mobile/executors/.

Never import node:* from tests/shared/ or tests/mobile/.

Placement under tests/shared/executors/ does not register an executor automatically. Explicitly check and update every compatible consumer:

• tests/desktop/consumer.ts
• tests/mobile/consumer.ts
• tests/electron/consumer.ts — also used by the Snap consumer

If a consumer cannot support the tests, add a documented SkipExecutor; do not leave scheduled tests without a matching handler, which fails at runtime with No handler found.

Mobile concerns to address in the plan:

• Memory — can the target device RAM hold the model? If not, propose a smaller model variant on mobile, or a SkipExecutor entry.
• Filesystem — node:fs is unavailable. Assets must be bundled via qvac-test.config.js → consumers.mobile.assets.patterns.
• Platform-specific limitations — known iOS/Android issues (OOM, missing native lib, backend unsupported). Add a SkipExecutor at the top of tests/mobile/consumer.ts with a clear reason.

If the feature cannot run on mobile at all, document the skip reason. Evaluate Electron/Snap compatibility independently rather than treating desktop-only coverage as automatic.