InfluxDB runtime

• Overview of Go runtime profiles
• Analyze the Go runtime trace
• View runtime configuration

A Go runtime profile is a collection of stack traces showing call sequences that led to instances of a particular event. InfluxDB provides profile data for the following events:

• blocks
• CPU usage
• memory allocation
• mutual exclusion (mutex)
• OS thread creation

When you send a profile request to InfluxDB, the samples the events on the runtime to collect stack traces and statistics (e.g., number of bytes of memory for heap allocation events). For some profiles, you can set the number of seconds that InfluxDB will collect profile data.

Once data collection is complete, InfluxDB returns the profile data. The default response format is a compressed protocol buffer in profile.proto format. profile.proto files are compatible with the and go tool pprof analysis tools. For some profiles, InfluxDB provides an alternative human-readable plain text format with comments that translate to function calls and line numbers, but the tools and profile.proto format offer the following advantages:

• Read profiles from disk or HTTP.
• Aggregate and compare multiple profiles of the same type.
• Analyze and filter profile data.
• Generate visualizations and reports.

Use the /debug/pprof InfluxDB endpoints to download all the profiles at once or request them individually.

• Profile all memory allocations
• Profile CPU
• Profile heap memory allocations
• Profile thread creation

To download all runtime profiles at once, use an HTTP client to send a GET request to the /debug/pprof/all endpoint. go tool pprof can’t fetch profiles directly from /debug/pprof/all.

InfluxDB returns a gzipped tar file that contains the following profiles in the profile.proto format:

• profiles/allocs.pb.gz:
• profiles/block.pb.gz: profile blocking operations
• profiles/cpu.pb.gz: (Optional) .
• profiles/goroutine.pb.gz: profile goroutines
• profiles/mutex.pb.gz:
• profiles/threadcreate.pb.gz: profile thread creation

Use an HTTP client like curl or wget to download profiles from /debug/pprof/all.

Example

# Use `curl` to download a `.tar.gz` of all profiles after 10 seconds of CPU sampling.
# Use `tar` to extract the profiles folder.
curl "http://localhost:8086/debug/pprof/all?cpu=10s" | tar -xz
# Analyze an extracted profile.
go tool pprof profiles/heap.pb.gz

Profile all memory allocations

GET http://localhost:8086/debug/pprof/allocs

# Analyze the profile in interactive mode.
go tool pprof http://localhost:8086/debug/pprof/allocs
# `pprof` returns the following prompt:
#   Entering interactive mode (type "help" for commands, "o" for options)
#   (pprof)
# At the prompt, get the top N memory allocations.
(pprof) top10

Profiles operations that led to blocking on synchronization primitives and caused Go to suspend goroutine’s execution.

GET http://localhost:8086/debug/pprof/block

# Analyze the profile in interactive mode.
go tool pprof http://localhost:8086/debug/pprof/block
# `pprof` returns the following prompt:
#   Entering interactive mode (type "help" for commands, "o" for options)
#   (pprof)
(pprof) top10

Profile CPU

Profiles program counters sampled from the execution stack. To download the profile, use an HTTP client to send a GET request to the /debug/pprof/profile endpoint. go tool pprof can’t fetch the CPU profile directly.

GET http://localhost:8086/debug/pprof/profile

Use an HTTP client like curl or wget to download the profile.

Example

Use the seconds query parameter to control the sampling duration.

/debug/pprof/profile?seconds=SECONDS returns the same CPU profile as /debug/pprof/all?cpu=DURATION.

Example

# Get the CPU profile after 10 seconds of sampling.
curl "http://localhost:8086/debug/pprof/profile?seconds=10" -o cpu
# Get all profiles after 10 seconds of CPU sampling.
curl "http://localhost:8086/debug/pprof/all?cpu=10s" -o all.tar.gz

Profiles all current goroutines.

GET http://localhost:8086/debug/pprof/goroutine

Example

# Analyze the profile in interactive mode.
go tool pprof http://localhost:8086/debug/pprof/goroutine
# `pprof` returns the following prompt:
#   Entering interactive mode (type "help" for commands, "o" for options)
#  At the prompt, get the top N entries.
(pprof) top10

Profile heap memory allocations

Profiles heap, or memory allocations for live objects.

GET http://localhost:8086/debug/pprof/heap

Example

# Analyze the profile in interactive mode.
go tool pprof http://localhost:8086/debug/pprof/heap
# `pprof` returns the following prompt:
#   Entering interactive mode (type "help" for commands, "o" for options)
#   (pprof)
(pprof) top10
# pprof displays the list:
#   Showing nodes accounting for 142.46MB, 85.43% of 166.75MB total
#   Dropped 895 nodes (cum <= 0.83MB)
#   Showing top 10 nodes out of 143

GET http://localhost:8086/debug/pprof/mutex

Example

Profile thread creation

Profiles operations that led to the creation of OS threads.

GET http://localhost:8086/debug/pprof/threadcreate

Example

# Analyze the profile in interactive mode.
go tool pprof http://localhost:8086/debug/pprof/threadcreate
# `pprof` returns the following prompt:
#   Entering interactive mode (type "help" for commands, "o" for options)
#   (pprof)
#  At the prompt, get the top N entries.
(pprof) top10

To trace execution events for InfluxDB, use the /debug/pprof/trace endpoint with go tool trace.

GET http://localhost:8086/debug/pprof/trace

Example

# Download the trace file.
curl http://localhost:8086/debug/pprof/trace -o trace.out
# Analyze the trace.
go tool trace ./trace.out

Generate a pprof-like profile from trace

You can use go tool trace to generate pprof-like profiles from a trace file and then analyze them with go tool pprof.

Example

# Generate a profile from the downloaded trace file.
go tool trace -pprof=PROFILE_TYPE ./trace.out > PROFILE_TYPE.pprof

Replace PROFILE_TYPE with one of the following :

• net: network blocking profile
• sync: synchronization blocking profile
• syscall: syscall blocking profile
• sched: scheduler latency profile

To view the command, arguments, and command-line variables that invoked InfluxDB, use the /debug/pprof/cmdline endpoint.

/debug/pprof/cmdline returns the command line invocation in plain text.

In InfluxDB v2.2+, you can view your active runtime configuration, including flags and environment variables. See how to view your runtime server configuration.