This ensures you can view traffic, metrics and graphs for resources deployed in a namespace with istio-injection=enabled
label.
If you would like to limit Prometheus to specific namespaces, set prometheus.prometheusSpec.ignoreNamespaceSelectors=true
. Once you do this, you will need to add additional configuration to continue to monitor your resources.
- Limiting Monitoring to Specific Namespaces by Setting ignoreNamespaceSelectors to True
- Monitoring Specific Namespaces: Create a Service Monitor or Pod Monitor
This limits monitoring to specific namespaces.
- From the Cluster Explorer, navigate to Installed Apps if Monitoring is already installed, or Charts in Apps & Marketplace
- If starting a new install, Click the rancher-monitoring chart, then in Chart Options click Edit as Yaml.
- If updating an existing installation, click on Upgrade, then in Chart Options click Edit as Yaml.
- Set
prometheus.prometheusSpec.ignoreNamespaceSelectors=true
- Complete install or upgrade
Result: Prometheus will be limited to specific namespaces which means one of the following configurations will need to be set up to continue to view data in various dashboards
- Monitoring specific namespaces: Add a Service Monitor or Pod Monitor in the namespace with the targets you want to scrape.
- Monitoring across namespaces: Add an
additionalScrapeConfig
to your rancher-monitoring instance to scrape all targets in all namespaces.
This option allows you to define which specific services or pods you would like monitored in a specific namespace.
The usability tradeoff is that you have to create the service monitor or pod monitor per namespace since you cannot monitor across namespaces.
- Run
kubectl create -f <name of service/pod monitor file>.yaml
if the file is stored locally in your cluster. - Or run
cat<< EOF | kubectl apply -f -
, paste the file contents into the terminal, then runEOF
to complete the command. - If starting a new install, Click the rancher-monitoring chart and scroll down to Preview Yaml.
- Run
kubectl label namespace <your namespace> istio-injection=enabled
to enable the envoy sidecar injection
Result: <your namespace>
can be scraped by prometheus.
This enables monitoring across namespaces by giving Prometheus additional scrape configurations.
The usability tradeoff is that all of Prometheus’ are maintained in a single Secret. This could make upgrading difficult if monitoring is already deployed with additionalScrapeConfigs before installing Istio.
- If starting a new install, Click the rancher-monitoring chart, then in Chart Options click Edit as Yaml.
- If updating an existing installation, click on Upgrade, then in Chart Options click Edit as Yaml.
- If updating an existing installation, click on Upgrade and then Preview Yaml.
- Set
prometheus.prometheusSpec.additionalScrapeConfigs
array to the Additional Scrape Config provided below. - Complete install or upgrade
Result: All namespaces with the istio-injection=enabled
label will be scraped by prometheus.
Additional Scrape Config
- job_name: 'istio/envoy-stats'
scrape_interval: 15s
metrics_path: /stats/prometheus
kubernetes_sd_configs:
- role: pod
- source_labels: [__meta_kubernetes_pod_container_port_name]
action: keep
regex: '.*-envoy-prom'
action: replace
regex: ([^:]+)(?::\d+)?;(\d+)
replacement: $1:15090
target_label: __address__
- action: labelmap
regex: __meta_kubernetes_pod_label_(.+)
- source_labels: [__meta_kubernetes_namespace]
action: replace
target_label: namespace
- source_labels: [__meta_kubernetes_pod_name]