A library that provides an embeddable, persistent key-value store for fast storage.
by facebookC++
Last 12 weeks · 188 commits
5 of 6 standards met
Summary: Leveled compaction combined with preserve_internal_time_seconds or preclude_last_level_data_seconds is a known-broken combination, but it is easy to enable by accident -- for example a fleet-wide config rollout that turns on the preserve time for CFs that happen to be leveled. Rather than degrading gracefully, it triggers an infinite kBottommostFiles compaction loop: a bottommost file gets marked, rewritten with no progress, and immediately re-marked, continuously rewriting the file and growing the MANIFEST so the file set never quiesces. This change makes such a misconfiguration a harmless no-op instead. Before this change, the marking logic (ComputeBottommostFilesMarkedForCompaction) marked a bottommost file whenever its largest seqno was below the oldest snapshot, without checking whether a compaction could actually zero that seqno. Recently written seqnos stay within the seqno->time "preserve" window and cannot be zeroed, so the rewrite never made progress. The same futile marking also occurred with user-defined timestamps when full_history_ts_low is unset. The underlying issue is that the "can this bottommost seqno be zeroed?" question was answered independently by the marking logic and by the compaction output path (CompactionIterator::PrepareOutput), and the two had drifted apart. With this change, both paths answer that question through one shared predicate (BottommostSeqnoCanBeZeroed), and the preserve-window boundary that marking needs is plumbed into VersionStorageInfo from DBImpl, mirroring the existing oldest-snapshot plumbing. Snapshot visibility stays a per-site check. On CFs without preserve/preclude the predicate is a no-op. Confirmed that caller invariants ensure that relocating the check in CompactionIterator::PrepareOutput() will not trip the ROCKSDB_LOG_FATAL. Test Plan: compaction_picker_test: new CompactionPickerTest BottommostFileNotMarkedWithinPreserveWindow / BottommostFileMarkedBelowPreserveWindow cover the preserve-window boundary; new CompactionPickerU64TsTest BottommostNotMarkedWithEmptyFullHistoryTsLowAndUnknownMaxTs covers the UDT edge case. tiered_compaction_test: new PrecludeBottommostLoopTest LeveledPreserveTimeNoBottommostLoop is an end-to-end regression for the loop and the DBImpl plumbing: a bottommost file with a nonzero largest seqno inside the preserve window, made a marking candidate via a snapshot release, must not be marked. It reproduces the loop (hangs) when the plumbing is neutralized and passes with the fix.
Summary: Summary RocksDB coroutine statistics are stored in , but their is thread-affine. A filesystem implementation may propagate the context while completing on another thread. Installing that context invokes , whose detects the thread hop and aborts. Before calling , shallow-copy the context and remove only the RocksDB coroutine-stats token. Filesystem implementations can still propagate the caller's other , while the thread-affine stats remain on the RocksDB thread. When the scope exits, assertions verify that the original context and stats token were restored before fallback accounting and coroutine suspension. Differential Revision: D117274942
Repository: facebook/rocksdb. Description: A library that provides an embeddable, persistent key-value store for fast storage. Stars: 32009, Forks: 6907. Primary language: C++. Languages: C++ (83.3%), Java (7.5%), C (2.5%), Python (2.3%), Starlark (1.9%). License: GPL-2.0. Homepage: http://rocksdb.org Topics: database, storage-engine. Latest release: v11.8.1 (2w ago). Open PRs: 100, open issues: 1522. Last activity: 2d ago. Community health: 75%. Top contributors: siying, igorcanadi, pdillinger, ajkr, yhchiang, riversand963, ltamasi, IslamAbdelRahman, hx235, cbi42 and others.