How-To: Set up Azure Monitor to search logs and collect metrics

• Enable Azure Monitor For containers in AKS
• Helm 3

1. Make sure that omsagents are running

You can use to enable prometheus metrics endpoint scrape.

If you installed Dapr to the different namespace, you need to change the array values. For example:

...
  prometheus-data-collection-settings: |-
    [prometheus_data_collection_settings.cluster]
        interval = "1m"
        monitor_kubernetes_pods_namespaces = ["dapr-system", "default"]
    [prometheus_data_collection_settings.node]
        interval = "1m"    
...

Apply config map:

1. Install Dapr with enabling JSON-formatted logs

1. Enable JSON formatted log in Dapr sidecar and add Prometheus annotations.

Add dapr.io/log-as-json: "true" annotation to your deployment yaml.

Example:

1. Go to Azure Monitor

Here is an example query, to parse JSON formatted logs and query logs from dapr system processes.

ContainerLog
| extend parsed=parse_json(LogEntry)
| project Time=todatetime(parsed['time']), app_id=parsed['app_id'], scope=parsed['scope'],level=parsed['level'], msg=parsed['msg'], type=parsed['type'], ver=parsed['ver'], instance=parsed['instance']
| sort by Time

1. Search metrics

This query, queries process_resident_memory_bytes Prometheus metrics for Dapr system processes and renders timecharts

References

• Azure Monitor Query

Last modified June 23, 2022: