Add Garnet RESP Command

Add a New Built-in RESP Command to Garnet

Step-by-step guide for implementing a new built-in RESP command in Garnet. This covers every file that must be created or modified, the tools that must be run, caveats discovered during implementation, and how to verify correctness.

Scope: Built-in commands that are part of the Garnet server (not custom extensions registered via REGISTERCS).

Overview: What Must Change

Adding a single new command touches at minimum these areas:

Step 1: Add the RespCommand Enum Value

File: libs/server/Resp/Parser/RespCommand.cs

The RespCommand enum is writes-first: write (mutating) commands occupy a dense, explicitly-numbered block immediately after NONE, followed by reads, scripts, and non-data:

Write commands:    APPEND = 1 ... BITOP_DIFF = 120   (EXPLICIT values; the ONLY persisted values)
Read commands:     BITCOUNT ... RISCAN               (auto-numbered; never persisted)
Script commands:   EVAL, EVALSHA                     (LastDataCommand = EVALSHA)
Non-key commands:  PING, SUBSCRIBE, etc.
Admin commands:    AUTH, CONFIG, etc.

Why writes-first: the write command's numeric value is persisted to disk (serialized into RespInputHeader.cmd in the AOF and streamed to replicas). Keeping writes in a dense, explicitly-numbered leading block makes those values stable — adding a read/admin command (which is never persisted) cannot shift them.

Read/write classification uses enum ranges (positional constants, not hardcoded values):

• RespCommandExtensions.FirstWriteCommand (APPEND=1) .. LastWriteCommand (BITOP_DIFF=120) → write
• FirstReadCommand .. LastReadCommand (= EVAL - 1) → read-only

Rules for adding a command:

• Write (mutating) command: APPEND it at the end of the write block, immediately before the // Read-only commands marker, with the next explicit value (e.g. MYCMD = 121,). Then update LastWriteCommand if it is no longer BITOP_DIFF. Never insert into the middle of the write block or change/reuse an existing value — persisted values must be append-only.
• Read-only / script / non-data command: add it in the appropriate (auto-numbered) section. These are never persisted, so their values may shift freely.
• Adding a write shifts the auto-numbered reads up by one — that is harmless (reads aren't persisted) and requires no other change.

Guardrails (do not bypass):

• PersistedEnumStabilityTests snapshots the write-block values and asserts the write range contains exactly the write commands. If you change an existing write value it fails — that is a breaking on-disk change requiring an AofHeader.AofHeaderVersion bump + a legacy remap in AofProcessor / LegacyRespCommand, not an edited expected value.
• If a new write command's value ever approaches the custom command range (0xFEFF), that is also caught by a test.

Boundary markers to watch (search for these comments):

APPEND = 1, // Note: FirstWriteCommand — keep as the first write command
BITOP_DIFF = 120, // Note: LastWriteCommand — append new write commands after this with the next value
EVALSHA, // Note: Update LastDataCommand if adding new data commands after this

Step 2: Add Command Parsing

File: libs/server/Resp/Parser/RespCommand.cs

Two parsing tiers exist:

Hash table path: RespCommandHashLookup (primary path for most commands)

The hash table in libs/server/Resp/Parser/RespCommandHashLookupData.cs provides O(1) lookup for all built-in commands. This is the recommended path for all new commands.

To add a new primary command, add one line to PopulatePrimaryTable() in RespCommandHashLookupData.cs:

Add("DELIFGREATER", RespCommand.DELIFGREATER);

To add a command with subcommands (e.g., MYPARENT SUBCMD):

1. Add the parent with hasSub: true in PopulatePrimaryTable():

   Add("MYPARENT", RespCommand.MYPARENT, hasSub: true);

2. Define the subcommand array in RespCommandHashLookupData.cs:

   private static readonly (string Name, RespCommand Command)[] MyparentSubcommands =
   [
       ("SUBCMD1", RespCommand.MYPARENT_SUBCMD1),
       ("SUBCMD2", RespCommand.MYPARENT_SUBCMD2),
   ];

3. Build the table in the static constructor in RespCommandHashLookup.cs:

   myparentSubTable = BuildSubTable(MyparentSubcommands, out myparentSubTableMask);

4. Wire it into LookupSubcommand() in RespCommandHashLookup.cs:

   RespCommand.MYPARENT => (myparentSubTable, myparentSubTableMask),

⚠️ Important: Use the exact wire-protocol spelling for the hash table name string. Some commands use hyphens (e.g., "SET-CONFIG-EPOCH" not "SETCONFIGEPOCH"). Check CmdStrings.cs for the canonical spelling.

⚠️ Convention: Define the command name string in libs/server/Resp/CmdStrings.cs and reference it from the parser, rather than using inline "..."u8 literals. This keeps command name strings centralized and reusable (e.g., for error messages).

// In CmdStrings.cs:
public static ReadOnlySpan<byte> DELIFGREATER => "DELIFGREATER"u8;

SIMD fast path: FastParseCommand() (optional, for hottest commands only)

Static Vector128<byte> patterns defined in libs/server/Resp/Parser/RespCommandSimdPatterns.cs that match the full RESP encoding (*N\r\n$L\r\nCMD\r\n) in a single 16-byte comparison. Use the RespPattern(argCount, "CMD") helper to create new patterns. Only needed for the most performance-critical commands with:

• Fixed argument count (single digit)
• Command names of 3-6 characters (total encoded length must fit in 16 bytes)

Most new commands should NOT be added here — the hash table + MRU cache provide excellent performance for all commands. Only add SIMD patterns if benchmarking shows the command is a bottleneck.

Step 3: Add Command Dispatch

File: libs/server/Resp/RespServerSession.cs

Three dispatch methods exist, and which one you use matters for latency tracking:

⚠️ WARNING: Do NOT add @slow-classified commands to ProcessBasicCommands or ProcessArrayCommands. This breaks latency tracking. If in doubt, use ProcessOtherCommands.

Pattern:

RespCommand.MYCMD => NetworkMYCMD(ref storageApi),

Add before the _ => ... fallthrough in the appropriate method.

Step 4: Implement the RESP Handler

New file or existing file in libs/server/Resp/

Command handlers are methods on the RespServerSession partial class:

private bool NetworkMYCMD<TGarnetApi>(ref TGarnetApi storageApi)
    where TGarnetApi : IGarnetApi
{
    // 1. Validate argument count
    if (parseState.Count != N)
        return AbortWithWrongNumberOfArguments(nameof(RespCommand.MYCMD));

    // 2. Validate other inputs (short-circuit before going to storage)
    var key = parseState.GetArgSliceByRef(0);
    // e.g., parse and validate optional flags, numeric arguments, etc.
    if (!parseState.TryGetInt(1, out var _))
    {
        WriteError(CmdStrings.RESP_ERR_GENERIC_VALUE_IS_NOT_INTEGER);
        return true;
    }

    // 3. Build input/output and call storage API
    // Note: To avoid double-parsing a parameter, you can pass a pre-parsed
    // value in the input struct's auxiliary arguments (e.g., input.arg1).
    var input = new StringInput(RespCommand.MYCMD, ref parseState, startIdx: 1);
    var output = GetStringOutput();
    var status = storageApi.MyOperation(key, ref input, ref output);

    // 4. Write RESP response
    if (status == GarnetStatus.OK)
    {
        ProcessOutput(output);
    }
    else
    {
        WriteError(CmdStrings.RESP_ERR_MY_MESSAGE);
    }

    return true;
}

Key patterns:

• Arguments: parseState.GetArgSliceByRef(i) returns ref PinnedSpanByte
• Input/Output: Instantiate StringInput/StringOutput (for string commands), ObjectInput/ObjectOutput (for object commands), or UnifiedInput/UnifiedOutput (for unified commands) before calling the storage API
• Response (happy path): Use ProcessOutput(output) in the common case — this handles writing the RESP response from the output struct
• Response (errors/special cases): Use RespServerSession extension methods (e.g., WriteError(...), WriteDirect(...), WriteInt64(...), etc.) — these handle SendAndReset() internally
• Error strings: Store as u8 literals in CmdStrings (e.g., CmdStrings.RESP_ERR_MY_MESSAGE) rather than inline
• Always return true — there are no partial executions

Object command RESP handler pattern

Object commands (Hash, List, Set, SortedSet) follow a similar pattern to string commands. The main difference is that the RESP handler uses ObjectInput/ObjectOutput with the appropriate operation enum and must handle WRONGTYPE errors:

File: libs/server/Resp/Objects/[ObjectName]Commands.cs (e.g., SortedSetCommands.cs)

private unsafe bool SortedSetAdd<TGarnetApi>(ref TGarnetApi storageApi)
    where TGarnetApi : IGarnetApi
{
    if (parseState.Count < 3)
        return AbortWithWrongNumberOfArguments("ZADD");

    var key = parseState.GetArgSliceByRef(0);

    var header = new RespInputHeader(GarnetObjectType.SortedSet) { SortedSetOp = SortedSetOperation.ZADD };
    var input = new ObjectInput(header, ref parseState, startIdx: 1);
    var output = GetObjectOutput();

    var status = storageApi.SortedSetAdd(key, ref input, ref output);

    switch (status)
    {
        case GarnetStatus.WRONGTYPE:
            while (!RespWriteUtils.TryWriteError(CmdStrings.RESP_ERR_WRONG_TYPE, ref dcurr, dend))
                SendAndReset();
            break;
        default:
            ProcessOutput(output.SpanByteAndMemory);
            break;
    }

    return true;
}

Key differences from string commands:

• Uses ObjectInput with a RespInputHeader(GarnetObjectType.XXX) and an operation enum (e.g., SortedSetOperation.ZADD)
• Must handle GarnetStatus.WRONGTYPE — object commands can fail if the key holds a different object type
• The actual data operation logic lives in libs/server/Objects/[ObjectName]/[ObjectName]ObjectImpl.cs, dispatched via the operation enum

For new object sub-operations: Add a value to the [ObjectName]Operation enum in libs/server/Objects/[ObjectName]/[ObjectName]Object.cs and handle it in the Operate method's switch statement.

⚠️ Persisted, append-only: the object operation enums (HashOperation, ListOperation, SetOperation, SortedSetOperation) have explicit values and are serialized to the AOF via RespInputHeader.SubId. Append the new operation at the end with the next explicit value; never insert into the middle, reorder, or change an existing value. PersistedEnumStabilityTests guards these values. The id occupies a full header byte, so up to 256 sub-operations per type are allowed.

Unified command note

Unified commands (EXISTS, DELETE, TYPE, TTL, EXPIRE, RENAME, etc.) are type-agnostic — they work on both raw string and object values. The RESP handler pattern is the same as string and object commands, but uses UnifiedInput/UnifiedOutput. The storage session layer uses the unified context (unifiedBasicContext), and the callbacks go in libs/server/Storage/Functions/UnifiedStore/.

Blocking command RESP handler pattern

Blocking commands (BLPOP, BRPOP, BLMOVE, BLMPOP, BZPOPMIN, BZPOPMAX, BZMPOP) follow a distinct pattern. They do not use ref storageApi and instead interact with the CollectionItemBroker (libs/server/Objects/ItemBroker/CollectionItemBroker.cs), which manages blocking/waiting behavior:

private unsafe bool SortedSetBlockingPop(RespCommand command)
{
    if (parseState.Count < 2)
        return AbortWithWrongNumberOfArguments(command.ToString());

    if (!parseState.TryGetTimeout(parseState.Count - 1, out var timeout, out var error))
        return AbortWithErrorMessage(error);

    var keysBytes = new byte[parseState.Count - 1][];
    for (var i = 0; i < keysBytes.Length; i++)
        keysBytes[i] = parseState.GetArgSliceByRef(i).ToArray();

    var result = storeWrapper.itemBroker.GetCollectionItemAsync(command, keysBytes, this, timeout).Result;

    if (!result.Found)
    {
        WriteNull();
    }
    else
    {
        // Write RESP response with result.Key, result.Item, result.Score, etc.
    }

    return true;
}

Key differences from regular commands:

• The dispatch in RespServerSession.cs does NOT pass ref storageApi: RespCommand.BZMPOP => SortedSetBlockingMPop(),
• The handler calls storeWrapper.itemBroker.GetCollectionItemAsync() which blocks (with timeout) until data is available
• No IGarnetApi method, no storage session method, and no RMW callbacks are needed for the blocking command itself (Steps 5-7 are skipped)
• The CollectionItemBroker is notified when data is added to a collection (e.g., ZADD calls itemBroker.HandleCollectionUpdate(key)), which wakes up blocked clients
• When adding a new blocking command, you must also update the TryGetResult method in CollectionItemBroker.cs to map your RespCommand to the correct GarnetObjectType and implement the retrieval logic

Steps 5–7: Storage Layer (skip for admin/non-key commands)

Note: Steps 5, 6, and 7 apply only to commands that read or write key-value data through the store (e.g., SET, GET, DELIFGREATER). Admin commands like DEBUG, PING, CONFIG, etc. handle their logic entirely in the RESP handler (Step 4) and do not need API interface methods, storage session ops, or RMW callbacks. Skip to Step 8 for those. Blocking commands (e.g., BZMPOP) also skip Steps 5-7 — see the blocking command pattern in Step 4.

Note on context types: The unified single-store has three context types: string context (for raw string commands like GET/SET), object context (for collection commands like ZADD/LPUSH), and unified context (for type-agnostic commands like EXISTS/DELETE/TTL/EXPIRE). Most new commands use either the string or object context — the unified context is only for commands that must work across both value types.

Step 5: Add API Interface Method

File: libs/server/API/IGarnetApi.cs

Add method signature to IGarnetApi (read-write) or IGarnetReadApi (read-only):

// String command:
GarnetStatus MyOperation(PinnedSpanByte key, ref StringInput input, ref StringOutput output);

// Object command:
GarnetStatus MyOperation(PinnedSpanByte key, ref ObjectInput input, ref GarnetObjectStoreOutput output);

// Unified command:
GarnetStatus MyOperation(PinnedSpanByte key, ref UnifiedInput input, ref UnifiedOutput output);

File: libs/server/API/GarnetApi*.cs

Add delegation in the GarnetApi partial struct. The implementation goes in the appropriate partial file based on the context type:

• GarnetApi.cs — string commands
• GarnetApiObjectCommands.cs — object commands
• GarnetApiUnifiedCommands.cs — unified commands

public GarnetStatus MyOperation(PinnedSpanByte key, ref StringInput input, ref StringOutput output)
    => storageSession.MyOperation(key, ref input, ref output);

⚠️ Caveat: GarnetApi is a generic partial struct: GarnetApi<TStringContext, TObjectContext, TUnifiedContext>. Always add your method to the correct partial file for the context type you're using.

Overloads for programmatic callers: In addition to the primary signature (used by the network handler), you can add simpler overloads for programmatic/embedded callers that avoid forcing them to create the Input/Output structs. For example:

public GarnetStatus MyOperation(PinnedSpanByte key, double val, out double output)

This overload internally creates the appropriate input/output structs and only returns the desired value to the caller, instead of writing to the output buffer.

Step 6: Implement Storage Session Layer

File: New or existing file in libs/server/Storage/Session/MainStore/ (for string-context ops), libs/server/Storage/Session/ObjectStore/ (for object-context ops), or libs/server/Storage/Session/UnifiedStore/ (for unified-context ops)

String command pattern

This layer wraps Tsavorite API calls. The network path uses a generic context parameter:

public GarnetStatus MyOperation<TStringContext>(PinnedSpanByte key, ref StringInput input, ref StringOutput output, ref TStringContext context)
    where TStringContext : ITsavoriteContext<...>
{
    var status = context.RMW((FixedSpanByteKey)key, ref input, ref output);
    if (status.IsPending)
        CompletePendingForSession(ref status, ref output, ref context);

    return GarnetStatus.OK;
}

Object and unified commands follow the same pattern — just substitute the appropriate context, input, and output types:

Programmatic overloads: You can also add simpler overloads for programmatic callers (see Step 5 note). These internally create the input/output structs and return only the desired value.

Object-specific: HandleCollectionUpdate

Object commands use the same pattern as above with ObjectInput/GarnetObjectStoreOutput and the object context.

⚠️ Caveat — HandleCollectionUpdate: If your object command modifies a collection (adds/removes elements), call itemBroker.HandleCollectionUpdate(key) after the store operation. This wakes up any clients blocked on that key (e.g., via BZPOPMIN). The actual data logic is implemented in the object class, not in the storage session.

Object implementation

File: libs/server/Objects/[ObjectName]/[ObjectName]ObjectImpl.cs

For object commands, the core logic lives in the object implementation. The Operate method in [ObjectName]Object.cs dispatches to implementation methods based on the operation enum:

case SortedSetOperation.ZADD:
    SortedSetAdd(ref input, ref output.SpanByteAndMemory);
    break;

The implementation methods in [ObjectName]ObjectImpl.cs directly manipulate the object's internal data structures (e.g., sortedSet, sortedSetDict for SortedSet).

Step 7: Add RMW/Read Callbacks (if applicable)

File: libs/server/Storage/Functions/MainStore/RMWMethods.cs (string commands) or libs/server/Storage/Functions/UnifiedStore/RMWMethods.cs (unified commands)

If your command uses RMW, you must handle these callbacks:

Add a case RespCommand.MYCMD: to each relevant switch statement.

For Read commands (MainStore): If your command uses Read (not RMW), the read response logic lives in libs/server/Storage/Functions/MainStore/PrivateMethods.cs — add a case to the CopyRespToWithInput method.

File: libs/server/Storage/Functions/MainStore/VarLenInputMethods.cs (string commands) or libs/server/Storage/Functions/UnifiedStore/VarLenInputMethods.cs (unified commands)

If your command writes a value, you must specify the value length:

⚠️ Caveat — RecordType: If your command creates records with a custom RecordType (e.g., for type discrimination), you can set it in InitialUpdater after record initialization:

var header = logRecord.RecordDataHeader;
header.RecordType = MyManager.MyRecordType;

This works because RecordDataHeader.RecordType has a setter that writes through a raw pointer. No Tsavorite infrastructure changes needed (despite the TODO comments in LogRecord.cs).

Step 8: Command Metadata Registration