Wally E2E

Device / overlay gotchas (0.5.1 + kit 0.20.28)

Public bottles never list neurt or qhexrt. That is the product, not a test gap. Overlay-rebuild the product binary (WALLY_APPLE_MLX_HOST=ON on Mac; ARM64 MSVC + QHexRT overlay on Snapdragon).

• CMAKE_PREFIX_PATH is not an overlay opt-in. Only WALLY_SDK_KIT makes e2e require neurt/qhexrt. An ambient overlay prefix from a previous rebuild will otherwise fail a public-bottle run.
• Binary assert: never nm | grep -q under pipefail (SIGPIPE → false FAIL). Stream strings -a / nm -a. Darwin MLX proof is mlx-swift_Cmlx.bundle next to product wally (nm -gU misses Swift host symbols). First C++ rac_plugin_register(mlx) logs -811; Swift callbacks then register MLX — noisy, not a miss.
• Windows ARM64 public/overlay kits have HAS_LLAMACPP FALSE. Do not require llamacpp in e2e. Overlay wally.exe listing only qhexrt (priority 150) is correct. On-disk GGUF (qwen3.5-2b) cannot run there.
• QHexRT generate needs QAIRT matching the device skel, not the overlay DLL set. Snapdragon X2 Elite / Hexagon v81: QNN_SDK_ROOT + ADSP_LIBRARY_PATH=%QNN_SDK_ROOT%\lib\hexagon-v81\unsigned, copy aarch64-windows-msvc QnnHtp.dll / QnnHtpPrepare.dll / QnnHtpV81Stub.dll / QnnHtpV81CalculatorStub.dll / QnnSystem.dll next to wally.exe. Overlay 2.47 DLLs vs device 2.41 skels fail; QAIRT 2.48 worked. Pass the *_HNPU directory (--engine qhexrt), not a GGUF. FastRPC openSession timeouts (~90s) then user-driver fallback are normal; a second generate while DSP is wedged fails with Skel failed to process context binary / 0x3ea — taskkill wally.exe and use a .bat with fully expanded ADSP_LIBRARY_PATH (nested %QNN_SDK_ROOT% in cmd /c "set A=…&& set B=%A%\…" does not expand).
• VS on the ARM64 box may be 2026 / 18 Community, not 2022: C:\Program Files\Microsoft Visual Studio\18\Community\VC\Auxiliary\Build\vcvarsarm64.bat. CMake/Ninja live under VS CMake extensions; they are not on default PATH.
• v0.20.28 Windows ARM64 public kit omits libcurl.lib. Copy from arm64-windows-static into the kit lib/ before linking (fixed in the SDK packager for the next kit; do not retag 0.20.28). Wally already links kit libcurl.lib when present.
• Published product bottles: macOS wally-$V-macos-arm64.tar.gz, Windows x86_64 zip, Windows arm64 zip, and Linux x86_64 wally-$V-linux-x86_64.tar.gz. NPU (NeuRT/QHexRT) is overlay-only on any platform.
• The private QHexRT overlay tarball used to ship zero skel files (only .dll/.lib, no .so/.cat) — rac-cli's own overlay build could not run qwen3.8-27b-1bit-npu (the Bonsai/Maple ternary decoder) out of the box; validating it required hand-copying librun_main_on_hexagon_skel.so
  • .cat in from the electron-qhexrt npm package as a workaround. Fixed in runanywhere-sdks' scripts/build/package-private-engine-overlay.sh (widened the copy filter and added a pass for dsp/win-arm64/). Wally itself never had the ADSP_LIBRARY_PATH bug the Electron binding had — fastrpc_win.cpp's exe_dir() fallback naturally resolves for wally.exe because dependent DLLs/skels are staged flat beside the executable by this repo's own packaging convention — but that protection is a property of the packaging layout, not of Wally's code, so it is not something to assume going forward. Always build a fresh overlay from the actual release script and run the ternary model against it after any SDK kit-pin bump that touches QHexRT — do not assume last time's manually-patched overlay is still representative of what a real user's overlay build produces.
• The private overlay tarball must be EXTRACTED ON TOP OF kit/, merging into the same directory tree (overlay/bin/* → kit/bin/, overlay/lib/* → kit/lib/, overlay/include/* → kit/include/, overlay/share/... → kit/share/...) — never kept as a separate sibling overlay/ directory fed to CMake via a second CMAKE_PREFIX_PATH entry. wally_stage_windows_runtime_dlls() (cmake/RunAnywhereSDK.cmake) only ever copies from ${RunAnywhere_LIBRARY_DIR}/../bin — i.e. kit/bin — so a same-named overlay/bin sitting next to kit/ is silently never consulted. Worse, this fails completely silently: the build succeeds, wally.exe links, and wally backends --json returns {"backends":[]} with no error naming QHexRT at all (find_library-style detection in RunAnywhereSDK.cmake just doesn't find kit/lib/rac_backend_qhexrt.lib because it was never copied there). If a fresh overlay build reports zero backends, check this BEFORE suspecting the overlay tarball's contents.
• qwen3.8-27b-1bit-npu's HostOpFailed had THREE compounding causes, found and fixed one at a time — a kit-pin bump to v0.20.31 alone was NOT enough; Wally needed its own additional fix (below) even with a perfectly merged overlay.
  1. The overlay-skel-files-never-shipped bug (above), fixed upstream in runanywhere-sdks' overlay packaging script.
  2. qhexrt::qnn::Backend::profile() — called (via the same engines/qhexrt/qhexrt_session.cpp this repo statically links, same as the Electron binding) to pick the v75/v79/v81 manifest directory before the manifest is even parsed — shared its device query with the code path that opens a real QNN HTP device, so the ternary decoder's host_only manifest paid for a live QNN device it never needed. Fixed in neurun v0.20.31 (Backend::profile() no longer shares ensure_device() with device()) — see that repo's qhexrt-profile-must-not-create-live-device KB finding.
  3. Wally-specific, and NOT fixed by the kit-pin bump alone: copy-overlay-dlls.cmake globbed *.dll only, so even a correctly merged overlay (per the bullet above) left the Bonsai skel's .so/ .cat sitting in kit/bin/ and NEVER staged next to wally.exe — the one place fastrpc_win.cpp's ADSP_LIBRARY_PATH ∪ exe_dir() search actually looks. Fixed by widening the glob to *.dll *.so *.cat. fastrpc_win.cpp's SET_PATH/GET_PATH both returning a non-zero rc (0x14/AEE_EUNSUPPORTED) is EXPECTED and HARMLESS on this driver (libcdsprpc 11.1.4 simply doesn't implement that control call — see that file's own header comment) — do not treat it as a symptom of anything. This was chased as a diagnostic signal once and wasted real device time; the only signal that matters is whether remote_handle64_open for the skel itself returns non-zero (0x80000406 = AEE_EUNABLETOLOAD, which that same file's header comment exhaustively catalogs the causes of — missing skel, missing/wrong/stale .cat, or — as this entry adds — the pair never being in the searched directory at all). Confirmed fixed end to end on a Snapdragon X2 Elite with all three fixes in place: wally run --engine qhexrt against qwen3.8-27b-1bit-npu opens the cDSP session and generates correctly ("The capital of France is Paris.", 0.105 tok/s, 12425 DSP linears).