The most important options that affects memtable behavior are:

  • : The factory object of memtable. By specifying factory object user can change the underlying implementation of memtable, and provide implementation specific options.
  • write_buffer_size: Size of a single memtable.
  • db_write_buffer_size: Total size of memtables across column families. This can be used to manage the total memory used by memtables.
  • max_write_buffer_number: The maximum number of memtables build up in memory, before they flush to SST files.The default implementation of memtable is based on skiplist. Other than the default memtable implementation, users can use other types of memtable implementation, for example HashLinkList, HashSkipList or Vector, to speed-up some queries.

HashSkiplist MemTable

As their names imply, HashSkipList organizes data in a hash table with each hash bucket to be a skip list, while HashLinkList organizes data in a hash table with each hash bucket as a sorted single linked list. Both types are built to reduce number of comparisons when doing queries. One good use case is to combine them with PlainTable SST format and store data in RAMFS.

The biggest limitation of the hash based memtables is that doing scan across multiple prefixes requires copy and sort, which is very slow and memory costly.

  • Memtable size exceed after a write.
  • Total memtable size across all column families exceed db_write_buffer_size, or write_buffer_manager signals a flush. In this scenario the largest memtable will be flushed.
  • Total WAL file size exceed . In this scenario the memtable with the oldest data will be flushed, in order to allow the WAL file with data from this memtable to be purged.As a result, a memtable can be flushed before it is full. This is one reason the generated SST file can be smaller than the corresponding memtable. Compression is another factor to make SST file smaller than corresponding memtable, since data in memtable is uncompressed.

Concurrent Insert

Without support of concurrent insert to memtables, concurrent writes to RocksDB from multiple threads will apply to memtable sequentially. Concurrent memtable insert is enabled by default and can be turn off via allow_concurrent_memtable_write option, although only skiplist-based memtable supports the feature.

In-place Update