Isc Mem Allocator

BIND 9 allocator wrapper (isc_mem / isc_mempool)

Source of truth: lib/isc/include/isc/mem.h (public API), lib/isc/mem.c (implementation), lib/isc/mem_p.h (private init/shutdown), lib/isc/jemalloc_shim.h (non-jemalloc fallback).

Big picture

isc_mem_t (an "mctx") is a thin, reference-counted accounting wrapper over jemalloc's non-standard API: mallocx / sdallocx / rallocx / sallocx. On systems without jemalloc, jemalloc_shim.h emulates those on top of malloc, recovering sizes from malloc_usable_size() / malloc_size(), or as a last resort by prepending a size_info header to every allocation.

Key consequences:

• Allocation never fails. OOM calls oom(), which writes a signal-safe report + backtrace to stderr and abort()s. Never write if (ptr == NULL) after an isc_mem_* call; there is no NULL return.
• Deallocation is size-hinted (sdallocx). Freeing with the wrong size is undefined behavior inside jemalloc, not just a stats bug.
• All contexts share jemalloc arenas (jemalloc_flags is currently always 0), so an mctx is an accounting domain, not a heap: it tracks who owes what, it does not partition memory.
• A global default context isc_g_mctx always exists (created by the library constructor isc__lib_initialize() → isc__mem_initialize() in lib/isc/lib.c). Use it only when nothing more specific fits.

The two allocation families — never mix them

Rules:

• Memory from isc_mem_get must be returned with isc_mem_put using the same mctx and the same size. Memory from isc_mem_allocate must go back via isc_mem_free. Crossing the families corrupts the per-context inuse counter (put charges the requested size, free charges the sallocx size), which detonates later as INSIST(isc_mem_inuse(ctx) == 0) when the context is destroyed. The header's ISC_ATTR_MALLOC_DEALLOCATOR_IDX attributes make compilers warn about some mismatches — treat those warnings as errors.
• Prefer the sized family whenever the caller knows the size (the common case: isc_mem_get(mctx, sizeof(*obj))). Use allocate/free only for variable-length data whose size is inconvenient to carry around.
• The c-prefixed variants are calloc-alikes: ISC_CHECKED_MUL(n, size) (overflow-fatal multiply) plus ISC__MEM_ZERO. ISC__MEM_ZERO is verbatim jemalloc's MALLOCX_ZERO (0x40), RUNTIME_CHECKed at startup. Free a cget'd array with isc_mem_cput(mctx, p, n, size) so the sizes match.
• Size 0 is legal and symmetric: both get and put bump 0 to sizeof(void *) (ADJUST_ZERO_ALLOCATION_SIZE).

The put/free macros NULL the pointer

isc_mem_put, isc_mem_cput, isc_mem_free, isc_mem_putanddetach, and isc_mempool_put are statement macros that end with (p) = NULL;. So:

• The pointer argument must be an assignable lvalue and must not have side effects (macro arguments are expanded more than once — never isc_mem_put(mctx, arr[i++], size)).
• After the call the variable is NULL — code relying on the old pointer value afterwards is a bug even though the memory "was just there".

isc_mem_putanddetach

For objects that hold a reference to their own mctx:

isc_mem_putanddetach(&obj->mctx, obj, sizeof(*obj));

frees obj and then drops the mctx reference in the safe order (equivalent to attach-to-local / detach-member / put / detach-local). This is the standard destructor idiom; use it instead of an open-coded put + detach.

Context lifecycle

• isc_mem_create("name", &mctx) — name is mandatory, copied, shows up in stats channel output and leak dumps. (The create macro also re-assigns isc__mem_malloc = mallocx; that is a deliberate link-order hack for jemalloc — see jemalloc issue #2566 — don't "clean it up".)
• Refcounted via ISC_REFCOUNT_IMPL: isc_mem_attach(src, &dst), isc_mem_detach(&mctx) (NULLs the pointer), isc_mem_ref/unref. The last detach destroys the context.
• Destruction asserts the books balance: INSIST(isc_mem_inuse(ctx) == 0) and that no pools remain. isc_mem_setdestroycheck(mctx, false) disables the leak check — almost never the right fix; a failing inuse INSIST means a leak or a family/size mismatch.
• isc_mem_checkdestroyed(stderr) (called by named/tests at shutdown) arms a library-shutdown check that every context was destroyed; it hits UNREACHABLE() otherwise, after dumping live contexts when debugging is on.
• isc__mem_shutdown() runs rcu_barrier() before checking, so RCU-deferred frees (call_rcu) are flushed first — deferred frees still count as live until the grace period runs.

Statistics and accounting internals

• inuse is striped per thread id: stat_s[ISC_TID_MAX + 1] cacheline-padded slots; ctx->stat = &stat_s[1] so isc_tid() == -1 (ISC_TID_UNKNOWN, i.e. threads outside the loopmgr) indexes stat[-1] legally. Updates are relaxed atomics on the caller's own stripe.
• A stripe can go negative (thread A frees what thread B allocated) — that's why stripes are signed. Only the sum (isc_mem_inuse(), which walks -1..isc_tid_count()) is meaningful.
• isc_mem_inuse() is O(threads) and iterates all stripes — fine for water-mark checks, don't put it in per-packet hot paths gratuitously.

Water marks / overmem

• isc_mem_setwater(mctx, hiwater, lowater) (0,0 or isc_mem_clearwater() disables). Used by the resolver/cache to bound cache memory.
• isc_mem_isovermem() is probabilistic: false below lowater, true above hiwater, and in between returns true with probability ramping linearly 0→1 (8-bit resolution, isc_random8()). This deliberately spreads cache cleaning over many inserts instead of a thundering herd at the mark — do not "fix" the randomness, and don't expect two consecutive calls to agree.

Returning memory to the OS

• Each thread counts bytes it frees (freed_bytes, thread-local); every 16 MiB it triggers mem_purge(): jemalloc arena.<all>.decay (or glibc malloc_trim(0)), rate-limited via CAS on last_purge to once per second globally.
• Init-time jemalloc tuning: background_thread = true, dirty_decay_ms = 10000 applied to existing and future arenas. Failures are ignored on purpose (the volumetric purge covers it).