Longhorn supports using more than one disk on the nodes to store the volume data.
By default, on the host will be used for storing the volume data. You can avoid using the default directory by adding a new disk, then disable scheduling for /var/lib/longhorn
.
To add a new disk for a node, head to the Node
tab, select one of the nodes, and select Edit Disks
in the dropdown menu.
To add any additional disks, you need to:
- Add the path of the mounted disk into the disk list of the node.
A certain amount of disk space can be reserved to stop Longhorn from using it. It can be set in the Space Reserved
field for the disk. It’s useful for the non-dedicated storage disk on the node.
The kubelet needs to preserve node stability when available compute resources are low. This is especially important when dealing with incompressible compute resources, such as memory or disk space. If such resources are exhausted, nodes become unstable. To avoid kubelet Disk pressure
issue after scheduling several volumes, by default, Longhorn reserved 30% of root disk space () to ensure node stability.
If you don’t want to use the original mount path of a disk on the node, you can use mount --bind
to create an alternative/alias path for the disk, then use it with Longhorn. Notice that soft link ln -s
won’t work since it will not get populated correctly inside the pod.
Nodes and disks can be excluded from future scheduling. Notice that any scheduled storage space won’t be released automatically if the scheduling was disabled for the node.
In order to remove a disk, two conditions need to be met:
- The scheduling for the disk must be disabled
- There is no existing replica using the disk, including any replicas in an error state. About how to evict replicas from disabled disks, refer to Select Disks or Nodes for Eviction.
Once those two conditions are met, you should be allowed to remove the disk.
There are two global settings affect the scheduling of the volume.
StorageMinimalAvailablePercentage
defines when a disk cannot be scheduled with more volumes. The default value is10
(%). The bigger value betweenMaximumStorage * StorageMinimalAvailablePercentage / 100
andMaximumStorage - ReservedStorage
will be used to determine if a disk is running low and cannot be scheduled with more volumes.
- Longhorn volumes can be bigger than the specified size, due to fact that the snapshot contains the old state of the volume.
- Longhorn does over-provisioning by default.