Troubleshooting the Istio CNI plugin

The Istio CNI plugin log provides information about how the plugin configures application pod traffic redirection based on .

The plugin runs in the container runtime process space, so you can see CNI log entries in the kubelet log. To make debugging easier, the CNI plugin also sends its log to the istio-cni-node DaemonSet.

The default log level for the CNI plugin is info. To get more detailed log output, you can change the level by editing the values.cni.logLevel installation option and restarting the CNI DaemonSet pod.

The CNI DaemonSet generates metrics, which can be used to monitor CNI installation, readiness, and race condition mitigation. Prometheus scraping annotations (, prometheus.io/path) are added to the istio-cni-node DaemonSet pod by default. You can collect the generated metrics via standard Prometheus configuration.

Readiness of the CNI DaemonSet indicates that the Istio CNI plugin is properly installed and configured. If Istio CNI DaemonSet is unready, it suggests something is wrong. Look at the istio-cni-node DaemonSet logs to diagnose. You can also track CNI installation readiness via the istio_cni_install_ready metric.

By default, the Istio CNI DaemonSet has enabled, which will evict a pod that was started before the CNI plugin was ready. To understand which pods were evicted, look for log lines like the following:

A common issue with the CNI plugin is that a pod fails to start due to container network set-up failure. Typically the failure reason is written to the pod events, and is visible via pod description:

If a pod keeps getting init error, check the init container istio-validation log for “connection refused” errors like the following:

The istio-validation init container sets up a local dummy server which listens on traffic redirection target inbound/outbound ports, and checks whether test traffic can be redirected to the dummy server. When pod traffic redirection is not set up correctly by the CNI plugin, the istio-validation init container blocks pod startup, to prevent traffic bypass. To see if there were any errors or unexpected network setup behaviors, search the istio-cni-node for the pod ID.