This ensures you can view traffic, metrics and graphs for resources deployed in a namespace with istio-injection=enabled label.

If you would like to limit Prometheus to specific namespaces, set prometheus.prometheusSpec.ignoreNamespaceSelectors=true. Once you do this, you will need to add additional configuration to continue to monitor your resources.

This limits monitoring to specific namespaces.

  1. From the Cluster Explorer, navigate to Installed Apps if Monitoring is already installed, or Charts in Apps & Marketplace
  2. If starting a new install, Click the rancher-monitoring chart, then in Chart Options click Edit as Yaml.
  3. If updating an existing installation, click on Upgrade, then in Chart Options click Edit as Yaml.
  4. Setprometheus.prometheusSpec.ignoreNamespaceSelectors=true
  5. Complete install or upgrade

Result: Prometheus will be limited to specific namespaces which means one of the following configurations will need to be set up to continue to view data in various dashboards

  • Monitoring specific namespaces: Add a Service Monitor or Pod Monitor in the namespace with the targets you want to scrape.
  • Monitoring across namespaces: Add an additionalScrapeConfig to your rancher-monitoring instance to scrape all targets in all namespaces.

This option allows you to define which specific services or pods you would like monitored in a specific namespace.

The usability tradeoff is that you have to create the service monitor or pod monitor per namespace since you cannot monitor across namespaces.

  1. Run kubectl create -f <name of service/pod monitor file>.yaml if the file is stored locally in your cluster.
  2. Or run cat<< EOF | kubectl apply -f -, paste the file contents into the terminal, then run EOF to complete the command.
  3. If starting a new install, Click the rancher-monitoring chart and scroll down to Preview Yaml.
  4. Run kubectl label namespace <your namespace> istio-injection=enabled to enable the envoy sidecar injection

Result: <your namespace> can be scraped by prometheus.

This enables monitoring across namespaces by giving Prometheus additional scrape configurations.

The usability tradeoff is that all of Prometheus’ are maintained in a single Secret. This could make upgrading difficult if monitoring is already deployed with additionalScrapeConfigs before installing Istio.

  1. If starting a new install, Click the rancher-monitoring chart, then in Chart Options click Edit as Yaml.
  2. If updating an existing installation, click on Upgrade, then in Chart Options click Edit as Yaml.
  3. If updating an existing installation, click on Upgrade and then Preview Yaml.
  4. Setprometheus.prometheusSpec.additionalScrapeConfigs array to the Additional Scrape Config provided below.
  5. Complete install or upgrade

Result: All namespaces with the istio-injection=enabled label will be scraped by prometheus.

Additional Scrape Config

  1. - job_name: 'istio/envoy-stats'
  2. scrape_interval: 15s
  3. metrics_path: /stats/prometheus
  4. kubernetes_sd_configs:
  5. - role: pod
  6. - source_labels: [__meta_kubernetes_pod_container_port_name]
  7. action: keep
  8. regex: '.*-envoy-prom'
  9. action: replace
  10. regex: ([^:]+)(?::\d+)?;(\d+)
  11. replacement: $1:15090
  12. target_label: __address__
  13. - action: labelmap
  14. regex: __meta_kubernetes_pod_label_(.+)
  15. - source_labels: [__meta_kubernetes_namespace]
  16. action: replace
  17. target_label: namespace
  18. - source_labels: [__meta_kubernetes_pod_name]