• Increase read amplification, significantly degrading read performance.The idea is to slow down incoming writes to the speed that the database can handle. However, sometimes the database can be too sensitive to a temporary write burst, or underestimate what the hardware can handle, so that you may get unexpected slowness or query timeouts.

    To find out whether your DB is suffering from write stalls, you can look at:

    • LOG file, which will contain info log when write stalls are triggered;
    • found in LOG file.Stalls may be triggered for the following reasons:

    • Too many level-0 SST files. When the number of level-0 SST files reaches level0_slowdown_writes_trigger, writes are stalled. When the number of level-0 SST files reaches , writes are fully stopped to wait for level-0 to level-1 compaction reduce the number of level-0 files. In these cases, you will get info logs in LOG file similar to

    Stopping writes because we have 20 level-0 files

    • Too many pending compaction bytes. When estimated bytes pending for compaction reaches soft_pending_compaction_bytes, writes are stalled. When estimated bytes pending for compaction reaches hard_pending_compaction_bytes, write are fully stopped to wait for compaction. In these cases, you will get info logs in LOG file similar to

    Stopping writes because of estimated pending compaction bytes 1000000000

    There are multiple options you can tune to mitigate write stalls. If you have some workload which can tolerant write stalls and some don't, you can set some writes to Low Priority Write to avoid stalling in those latency-critical writes.

    If write stalls are triggered by pending flushes, you can try:

    • Increase max_background_flushes to have more flush threads.
    • Increase max_background_compactions to have more compaction threads.

    • Increase to have large memtable, to reduce write amplification.
    • Increase min_write_buffer_number_to_merge.You can also set stop/slowdown triggers and pending compaction bytes limits to huge number to avoid hitting write stall. Also take a look at "What's the fastest way to load data into RocksDB?" in our if you are bulk loading data to RocksDB.