PersistentPodState

    With the development of cloud native, more and more companies start to deploy stateful services (e.g., Etcd, MQ) using Kubernetes. K8S StatefulSet is a workload for managing stateful services, and it considers the deployment characteristics of stateful services in many aspects. However, StatefulSet persistent only limited pod state, such as Pod Name is ordered and unchanging, PVC persistence, and can not cover other states, e.g. Pod IP retention, priority scheduling to previously deployed Nodes, etc. Typical Cases:

    • Service Discovery Middleware services are exceptionally sensitive to the Pod IP after deployment, requiring that the IP cannot be changed.

    For detailed design, please refer to: PPS Proposal.

    Some common PersistentPodState can be generated by annotation to satisfy most of the scenarios. For some complex scenarios, you can use PersistentPodState CRD to define them directly.

    Define PersistentPodState CRD

    IP Retention Practice

    “IP Retention” should be a common requirement for K8S deployments of stateful services. It does not mean “Specified Pod IP”, but requires that the Pod IP does not change after the first deployment, either by service release or by machine eviction. To achieve this, we need the K8S network component to support Pod IP retention and the ability to keep the IP as unchanged as possible. In this article, we have modified the Host-local plugin in the flannel network component so that it can achieve the effect of keeping the Pod IP unchanged under the same Node. Related principles will not be stated here, please refer to the code: .

    1. Deploy stateful service echoserver, declaring “IP Retention” via annotations, as follows:

    2. According to the above configuration, kruise automatically generates PersistentPodState and records the node status of the first deployment of Pod in PersistentPodState.Status.

    3. After Pod rebuild due to service release or Node eviction, etc., kruise injects the recorded Pod node information into Pod NodeAffinity, which in turn enables the Pod IP to remain unchanged, as follows: