rocksdb

Summary Add a callback-based to the C API, enabling language bindings to plug in custom memory allocators (mimalloc, tcmalloc, snmalloc) at runtime without requiring them to be linked at RocksDB build time. Resolves #14367 Motivation The C API currently only exposes . On Windows (MSVC), jemalloc is not supported, and the CRT allocator never decommits freed heap segments — causing unbounded memory growth (#4112, #12364). Language bindings (Rust, Go, Python) have no way to plug in an alternative. Implementation Follows the exact callback pattern used by , , and : struct in inheriting Function pointers: , , (optional), (optional) + for user-managed lifetime Wrapped in via existing Changes : Add declaration : Add proxy class and factory function : Add test creating a custom allocator with malloc/free callbacks, wiring it to an LRU cache : Release note Test Plan Added test in Test creates allocator → sets on LRU cache options → creates cache → destroys in order Verifies no crashes or leaks through the callback dispatch path

Summary The C API currently only exposes for creating a . There is no way to create a custom, user-defined through the C API using function pointers — even though the C++ API fully supports this via subclassing . This prevents language bindings (Rust, Go, Python, Node.js) from plugging in alternative allocators like mimalloc, tcmalloc, or snmalloc — which is critical on Windows where jemalloc is not supported. Motivation The Windows memory problem On Windows (MSVC), RocksDB links against the CRT allocator (/). The Windows CRT never decommits freed heap segments — once committed virtual memory pages are allocated, marks them as available but does not call . This means: Any large allocation (e.g., iterating over a column family) permanently inflates the process's private bytes Block cache evictions free memory internally but the OS-level committed memory never shrinks Over time, processes using RocksDB on Windows exhibit unbounded memory growth This is the root cause behind #4112 (189 comments), #12364, and #12579. On Linux, jemalloc or tcmalloc solve this by returning memory to the OS via . But: jemalloc does not support Windows MSVC ( explicitly excludes ) tcmalloc requires which doesn't exist on Windows mimalloc (Microsoft's own allocator) works on Windows and returns memory, but there's no way to plug it into RocksDB through the C API Demand from language bindings The C API is the foundation for all non-C++ RocksDB bindings: Rust (): Issues rust-rocksdb#749, rust-rocksdb#926, rust-rocksdb#260 request custom cache/allocator support Go (/): No allocator control available Java (): Users in #4112 report memory growth with JNI, resort to hacks Python (): No allocator control All of these would benefit from a callback-based in the C API. Proposed API Add to : This follows the same callback + state pattern already used throughout the C API for comparators, merge operators, compaction filters, etc. Implementation sketch Add to : Scope Covers allocations that go through : Block cache (typically the largest memory consumer) Compressed block cache Blob cache Does not** cover iterator temporary buffers, compaction buffers, or memtable allocations (unless is configured with the same cache). A follow-up effort to route more internal allocations through would further improve control. Related issues #4112 — Memory grows without limit (189 comments) #12364 — Unexplained sudden increase in memory usage #12579 — High Memory Usage / LRU cache size not respected #1442 — RocksDB shouldn't determine at build time whether to use jemalloc/tcmalloc #4437 — Original PR introducing CacheAllocator (now MemoryAllocator) Happy to submit a PR implementing this if there's interest.