The bitmap serialization format in Doris is explained as follows:
C++ serialization and deserialization examples are in the BitmapValue::write()
method in be/src/util/bitmap_value.h
and the Java examples are in the serialize()
method in fe/fe-common/src/main/java/org/apache/doris/common/io/BitmapValue.java
.
HLL format description
Value | Description |
---|---|
0 | HLL_DATA_EMPTY, empty HLL, the following data part is empty, the entire serialization result is only one byte |
1 | HLL_DATA_EXPLICIT, the next byte is explicit The number of data blocks, followed by multiple data blocks, each data block is composed of 8 bytes in length and data, |
2 | HLL_DATA_SPARSE, only non-zero values are stored, the next 4 bytes indicate the number of registers, and there are multiple register structures in the back. Each register is composed of the index of the first 2 bytes and the value of 1 byte |
3 | HLL_DATA_FULL, which means that all 16 1024 registers have values, followed by 16 1024 bytes of value data |
C++ serialization and deserialization examples are in the serialize()
deserialize()
method of be/src/olap/hll.h
, and the Java examples are in the serialize()
method in fe/fe-common/src/main/java/org/apache/doris/common/io/hll.java
.