Corvus Buffer And Pooling

Buffer Management and Pooling Patterns

The Three-Tier Pooling Hierarchy

All temporary buffers in this codebase follow a strict three-tier allocation strategy. Use the cheapest tier that fits.

Tier 1: stackalloc           — ≤ threshold bytes, zero cost, no cleanup needed
Tier 2: ArrayPool<T>.Shared  — larger buffers, must return in finally block
Tier 3: [ThreadStatic] cache — expensive-to-create objects (writers, workspaces), reused per thread

Tier 1 + 2: The stackalloc / ArrayPool Rent Pattern

This is the single most common pattern in the codebase. Every temporary buffer follows it exactly.

byte[]? rentedArray = null;

Span<byte> buffer = length <= JsonConstants.StackallocByteThreshold
    ? stackalloc byte[JsonConstants.StackallocByteThreshold]
    : (rentedArray = ArrayPool<byte>.Shared.Rent(length));

try
{
    DoWork(buffer.Slice(0, length));
}
finally
{
    if (rentedArray != null)
    {
        ArrayPool<byte>.Shared.Return(rentedArray);
    }
}

Threshold Constants

Defined in Common/src/System/Text/Json/JsonConstants.cs:

Use NonRecursive variants only when you can prove the call site is not recursive — otherwise stack overflow is possible.

Rules (non-negotiable)

1. Declare rentedArray before the ternary — it must be in scope for the finally block
2. Use the named constant, not a magic number, for both the threshold check and the stackalloc size
3. Always slice to length — ArrayPool.Rent() returns an array that may be larger than requested
4. Always use try/finally to guarantee the rented array is returned
5. For fixed-size buffers always ≤ threshold (e.g., a 128-byte scratch buffer), plain stackalloc without pool fallback is acceptable

Sizing the buffer: GetMaxByteCount, not GetByteCount

When the length that sizes a transient UTF-8 scratch buffer comes from a string/char span, size it with Encoding.UTF8.GetMaxByteCount(chars.Length) (a multiply — chars.Length * 3 + 3), not Encoding.UTF8.GetByteCount(chars) (a full transcoding scan of every code point). GetMaxByteCount returns a safe upper bound, so the buffer is never under-sized; the exact filled length is whatever the subsequent Encoding.UTF8.GetBytes(chars, buffer) (or your assemble routine) returns, and you slice the buffer by that return value. The pattern is already "rent ≥ requested, then slice to actual", so an over-estimate of a few bytes costs nothing and skips the scan. Assembling several parts: sum each part's GetMaxByteCount plus exact-width separators.

// ✅ multiply, not a scan — buffer is an upper bound; `written` is the exact length the rest of the code uses
int max = Encoding.UTF8.GetMaxByteCount(text.Length);
byte[]? rented = max > Threshold ? ArrayPool<byte>.Shared.Rent(max) : null;
Span<byte> buffer = rented ?? stackalloc byte[Threshold];
int written = Encoding.UTF8.GetBytes(text, buffer);
Use(buffer[..written]);   // never the `max`

A public "how big a buffer do I need" helper that uses this should be named for the upper bound it returns (GetMaxEncodedLength, not GetEncodedLength) and documented as a safe size, not an exact count — otherwise a caller may trust it as exact. Pair it with a writer that reports what it actually wrote (EncodeToUtf8(out written)), so the caller sizes from the bound and then trims to the truth.

GetByteCount (exact) is still required — do NOT switch these to GetMaxByteCount: an exact single-shot output allocation (new byte[total] where total is the precise serialized size, e.g. WorkflowPackage.PackPooled), a structural length field written into a format (a ushort entry-name length), an offset you then copy at (dest[exactPrefixLen..]), or IBufferWriter.AppendSpan(n) / GetSpan(n)-style APIs that commit exactly n (over-sizing commits uninitialised trailing bytes). The rule is: transient scratch sliced by the actual written length → GetMaxByteCount; a value that is itself exact output, a committed length, or a copy offset → GetByteCount.

char buffer variant

char[]? rentedChars = null;

Span<char> charBuffer = length <= JsonConstants.StackallocCharThreshold
    ? stackalloc char[JsonConstants.StackallocCharThreshold]
    : (rentedChars = ArrayPool<char>.Shared.Rent(length));

try
{
    DoWork(charBuffer.Slice(0, length));
}
finally
{
    if (rentedChars != null)
    {
        ArrayPool<char>.Shared.Return(rentedChars);
    }
}

Tier 2: ArrayPool Usage in Core Types

Several core types rent directly from ArrayPool<T>.Shared:

When writing Dispose logic for pooled types, use Interlocked.Exchange(ref _array, null) to prevent double-return.

Tier 3: Thread-Local Caches

For objects that are expensive to construct but frequently needed, the codebase uses [ThreadStatic] caches with a depth-counter pattern:

[ThreadStatic]
private static ThreadLocalState? t_threadLocalState;

public static ExpensiveObject Rent(...)
{
    ThreadLocalState state = t_threadLocalState ??= new();
    if (state.RentedCount++ == 0)
    {
        // First (non-recursive) call — return the cached instance
        state.CachedObject.Reset(...);
        return state.CachedObject;
    }
    // Recursive call — allocate a fresh instance
    return new ExpensiveObject(...);
}

public static void Return(ExpensiveObject obj)
{
    ThreadLocalState state = t_threadLocalState!;
    if (--state.RentedCount == 0)
    {
        state.CachedObject.ResetAllStateForCacheReuse();
        // Cached object stays in thread-local for next Rent
    }
    else
    {
        obj.Dispose(); // Recursive instance — dispose immediately
    }
}

Three caches follow this pattern: