The in-memory index holds all the keys in a data structure, along with pointers to the on-disk data (the values). Each key in the B-tree may contain multiple pointers, pointing to different versions of its values. The theoretical memory consumption of the in-memory index can hence be approximated with the formula:

where c1 is the key metadata overhead and c2 is the version metadata overhead.

Page cache memory is managed by the operating system and is not covered in detail in this document.

etcd version- git head

GCE n1-standard-2 machine type

We calculate the memory usage consumption via the Go runtime.ReadMemStats. We calculate the total allocated bytes difference before creating the index and after creating the index. It cannot perfectly reflect the memory usage of the in-memory index itself but can show the rough consumption pattern.

Based on the result, we can calculate c1=120bytes, c2=30bytes. We only need two sets of data to calculate c1 and , since they are the only unknown variable in the formula. The c1=120bytes and c2=30bytes are the average value of the 4 sets of c1 and we calculated. The key metadata overhead is still relatively nontrivial (50%) for small key-value pairs. However, this is a significant improvement over the old store, which had at least 1000% overhead.

The overall memory usage captures how much RSS etcd consumes with the storage. The value size should have very little impact on the overall memory usage of etcd, since we keep values on disk and only retain hot values in memory, managed by the OS page cache.