Doca Gpunetio Ib Write Lat

DOCA GPUNetIO ib_write_lat

Where to start: This is a tool skill for the GPUNetIO- flavored ib_write_lat benchmark shipped under doca/tools/gpunetio_ib_write_lat/ (a client + server pair, built from source against the installed DOCA via meson). It measures the latency of an RDMA WRITE work request when the WR is posted from a CUDA kernel through the doca-gpunetio device-side surface, in a ping-pong cadence. Open TASKS.md and start at ## configure for the GPU-NIC pairing precondition and the build pattern; jump to ## run for the single-iteration smoke flow. Open CAPABILITIES.md when the question is what this tool actually measures, how it differs from the GPI sister tool on the same physical operation, or how to interpret the half-iter / full-iter / CUDA-side usec output and the median / p99 / jitter characterization. If DOCA is not installed yet, route to doca-setup first; if the user is still deciding between GPUNetIO and GPI as a programming surface, the picture in ../../libs/doca-gpunetio/CAPABILITIES.md#capabilities-and-modes and ../../libs/doca-gpi/CAPABILITIES.md#capabilities-and-modes is the first stop.

Example questions this skill answers well

The CLASSES of doca-gpunetio-ib-write-lat questions this skill is built to answer, each with one worked example. The class is the load-bearing piece; the worked example is one instance.

• "What GPU-init RDMA-WRITE latency / jitter can the GPUNetIO path deliver for a real-time / control-loop workload?" — worked example: "measure per-iteration WRITE latency between two hosts with an H100 + ConnectX-7 on each side, target the median and the p99 separately". Answered by the GPU-NIC pairing precondition in CAPABILITIES.md ## Capabilities and modes
  • the bring-up flow in TASKS.md ## configure + TASKS.md ## run.
• "This is the GPUNetIO tool — how does the latency number differ from the GPI programming surface?" — worked example: "the team is using GPI; should I expect GPUNetIO to beat / tie / lose vs GPI?". Answered by the "same physical operation, different runtime framework" rule in CAPABILITIES.md ## Capabilities and modes
  • the cross-link to the GPI library skill ../../libs/doca-gpi/CAPABILITIES.md (note: doca/tools/ ships no GPI ib_write_lat benchmark binary — GPI is a programming surface, not a shipped benchmark tool).
• "Median vs p99 vs jitter — which one is the actual answer for a real-time control loop?" — worked example: "my control loop has a deadline; the median is well under the budget but p99 spikes; do I quote the median or the p99?". Answered by the median-vs-p99-vs-jitter rule in CAPABILITIES.md ## Observability
  • the eval-loop overlay in TASKS.md ## test.
• "What is the latency-vs-batching trade-off specific to GPU-init RDMA?" — worked example: "my CUDA kernel could batch multiple WRs to amortize the GPU-side overhead; what does that buy me on latency vs what does it cost me?". Answered by the latency-vs-batching trade-off in CAPABILITIES.md ## Capabilities and modes.
• "What version of DOCA + CUDA Toolkit do I need for this binary to build and run?" — worked example: "my install has DOCA at one semver and CUDA at another; will the ToT-shipped gpunetio_ib_write_lat even link?". Answered by the version overlay in CAPABILITIES.md ## Version compatibility.
• "How do I read the half-iter / full-iter / CUDA-side usec columns?" — worked example: "the binary printed half-iter, full-iter, and a CUDA-side number — what is the right column to quote for one-way latency vs round-trip vs cross-check?". Answered by the column- semantics rule in CAPABILITIES.md ## Observability.

Audience

This skill serves external developers and performance engineers who need a reproducible measurement of the latency of an RDMA WRITE WR when the WR is posted from a CUDA kernel through doca-gpunetio, on the user's actual install and GPU-NIC pair. Concretely:

• A developer designing a GPU-resident real-time control loop and deciding whether the GPUNetIO path's tail latency fits the deadline.
• A platform operator validating a tuning change (NUMA pinning, GPU PCIe placement, IB device choice, GID index, NIC firmware burn) by re-running this benchmark against the new state.
• An SRE / performance engineer producing a "this is the GPUNetIO-driven WRITE latency on this GPU-NIC pair today, with median + p99 + jitter" artifact downstream consumers can cite.
• An AI agent answering "is the doca-gpunetio latency budget acceptable for this real-time workload class" honestly — with measured numbers, the build + invocation that produced them, and the GPU + NIC + DOCA version that scopes them — rather than guessing.

It is not for users debugging the doca-gpunetio library itself (route to ../../libs/doca-gpunetio/SKILL.md), and not a substitute for the perftest upstream ib_write_lat (which measures CPU-initiated WRITE latency).

Language scope

The doca-gpunetio-ib-write-lat tool is shipped as C plus CUDA .cu translation units under doca/tools/gpunetio_ib_write_lat/, split into a client/ subtree, a server/ subtree, and a common/ subtree shared between them (per the verified file layout: client/{main.c,perftest.{c,h},meson.build}, server/{main.c,perftest.{c,h},meson.build}, common/{common.c,common.h,kernel.cu}). The host-side build is meson against the installed DOCA pkg-config modules (doca-gpunetio, doca-rdma, doca-common, plus the CUDA Toolkit dependency); the device-side build is nvcc against the DOCA GPU NetIO device-side header set. There is no Python / Rust / Go binding — the tool is a pair of CLI binaries.

When to load this skill

Load this skill when the user is — or the agent needs to — build and run the gpunetio_ib_write_lat client + server on real hosts with DOCA installed plus a CUDA Toolkit matched to the DOCA install, and a GPU + IB device pair on each host's PCIe topology. Concretely:

• Measuring kernel-initiated RDMA WRITE latency between two hosts (or a host and a BlueField DPU) with the GPUNetIO surface.
• Characterizing tail latency (p99 / p99.9) and jitter for a real-time / control-loop workload class.
• Deciding whether the GPUNetIO path is the right runtime surface for a class of workload vs the GPI programming surface (the doca-gpi library — doca/tools/ ships no GPI benchmark binary) or the classic CPU-initiated perftest path.
• Capturing a documented baseline (build + invocation + DOCA version + GPU + NIC + as-deployed environment + numbers) for later regression hunts.
• Diagnosing a build / link / run failure that surfaces the GPUNetIO + RDMA bring-up sequence under this tool's shipped scaffolding.

Do not load this skill for general DOCA orientation, library API work, or installation. For those, use doca-public-knowledge-map, ../../libs/doca-gpunetio/SKILL.md, or doca-setup. Do not load it for application-level real-time deadline analysis — this benchmark measures the WR latency through GPUNetIO, not the user's full pipeline.