27.7. tracemalloc — 跟踪内存分配

源代码：

The tracemalloc module is a debug tool to trace memory blocks allocated by Python. It provides the following information:

Traceback where an object was allocated
Statistics on allocated memory blocks per filename and per line number: total size, number and average size of allocated memory blocks
Compute the differences between two snapshots to detect memory leaks

To trace most memory blocks allocated by Python, the module should be started as early as possible by setting the PYTHONTRACEMALLOC environment variable to , or by using tracemalloc command line option. The tracemalloc.start() function can be called at runtime to start tracing Python memory allocations.

By default, a trace of an allocated memory block only stores the most recent frame (1 frame). To store 25 frames at startup: set the environment variable to 25, or use the -X tracemalloc=25 command line option.

显示内存分配最多的10个文件：

Python测试套件的输出示例：

[ Top 10 ]
<frozen importlib._bootstrap>:716: size=4855 KiB, count=39328, average=126 B
<frozen importlib._bootstrap>:284: size=521 KiB, count=3199, average=167 B
/usr/lib/python3.4/collections/__init__.py:368: size=244 KiB, count=2315, average=108 B
/usr/lib/python3.4/unittest/case.py:381: size=185 KiB, count=779, average=243 B
/usr/lib/python3.4/unittest/case.py:402: size=154 KiB, count=378, average=416 B
/usr/lib/python3.4/abc.py:133: size=88.7 KiB, count=347, average=262 B
<frozen importlib._bootstrap>:1446: size=70.4 KiB, count=911, average=79 B
<frozen importlib._bootstrap>:1454: size=52.0 KiB, count=25, average=2131 B
<string>:5: size=49.7 KiB, count=148, average=344 B
/usr/lib/python3.4/sysconfig.py:411: size=48.0 KiB, count=1, average=48.0 KiB

We can see that Python loaded 4855 KiB data (bytecode and constants) from modules and that the module allocated 244 KiB to build namedtuple types.

See for more options.

27.7.1.2. 计算差异

获取两个快照并显示差异：

import tracemalloc
tracemalloc.start()
# ... start your application ...
snapshot1 = tracemalloc.take_snapshot()
# ... call the function leaking memory ...
snapshot2 = tracemalloc.take_snapshot()
top_stats = snapshot2.compare_to(snapshot1, 'lineno')
print("[ Top 10 differences ]")
for stat in top_stats[:10]:
    print(stat)

Example of output before/after running some tests of the Python test suite:

We can see that Python has loaded 8173 KiB of module data (bytecode and constants), and that this is 4428 KiB more than had been loaded before the tests, when the previous snapshot was taken. Similarly, the module has cached 940 KiB of Python source code to format tracebacks, all of it since the previous snapshot.

If the system has little free memory, snapshots can be written on disk using the Snapshot.dump() method to analyze the snapshot offline. Then use the method reload the snapshot.

27.7.1.3. Get the traceback of a memory block

Code to display the traceback of the biggest memory block:

import tracemalloc
# Store 25 frames
tracemalloc.start(25)
# ... run your application ...
snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics('traceback')
# pick the biggest memory block
stat = top_stats[0]
print("%s memory blocks: %.1f KiB" % (stat.count, stat.size / 1024))
for line in stat.traceback.format():
    print(line)

Example of output of the Python test suite (traceback limited to 25 frames):

903 memory blocks: 870.1 KiB
  File "<frozen importlib._bootstrap>", line 716
  File "<frozen importlib._bootstrap>", line 1036
  File "<frozen importlib._bootstrap>", line 934
  File "<frozen importlib._bootstrap>", line 1068
  File "<frozen importlib._bootstrap>", line 619
  File "<frozen importlib._bootstrap>", line 1581
  File "<frozen importlib._bootstrap>", line 1614
  File "/usr/lib/python3.4/doctest.py", line 101
    import pdb
  File "<frozen importlib._bootstrap>", line 284
  File "<frozen importlib._bootstrap>", line 938
  File "<frozen importlib._bootstrap>", line 1068
  File "<frozen importlib._bootstrap>", line 619
  File "<frozen importlib._bootstrap>", line 1581
  File "<frozen importlib._bootstrap>", line 1614
  File "/usr/lib/python3.4/test/support/__init__.py", line 1728
    import doctest
  File "/usr/lib/python3.4/test/test_pickletools.py", line 21
    support.run_doctest(pickletools)
  File "/usr/lib/python3.4/test/regrtest.py", line 1276
  File "/usr/lib/python3.4/test/regrtest.py", line 976
    display_failure=not verbose)
  File "/usr/lib/python3.4/test/regrtest.py", line 761
    match_tests=ns.match_tests)
  File "/usr/lib/python3.4/test/regrtest.py", line 1563
    main()
  File "/usr/lib/python3.4/test/__main__.py", line 3
    regrtest.main_in_temp_cwd()
  File "/usr/lib/python3.4/runpy.py", line 73
    exec(code, run_globals)
  File "/usr/lib/python3.4/runpy.py", line 160
    "__main__", fname, loader, pkg_name)

We can see that the most memory was allocated in the module to load data (bytecode and constants) from modules: 870.1 KiB. The traceback is where the importlib loaded data most recently: on the import pdb line of the module. The traceback may change if a new module is loaded.

27.7.1.4. Pretty top

Code to display the 10 lines allocating the most memory with a pretty output, ignoring <frozen importlib._bootstrap> and <unknown> files:

Python测试套件的输出示例：

Top 10 lines
#1: Lib/base64.py:414: 419.8 KiB
    _b85chars2 = [(a + b) for a in _b85chars for b in _b85chars]
#2: Lib/base64.py:306: 419.8 KiB
    _a85chars2 = [(a + b) for a in _a85chars for b in _a85chars]
#3: collections/__init__.py:368: 293.6 KiB
#4: Lib/abc.py:133: 115.2 KiB
    cls = super().__new__(mcls, name, bases, namespace)
#5: unittest/case.py:574: 103.1 KiB
    testMethod()
#6: Lib/linecache.py:127: 95.4 KiB
    lines = fp.readlines()
#7: urllib/parse.py:476: 71.8 KiB
    for a in _hexdig for b in _hexdig}
#8: <string>:5: 62.0 KiB
#9: Lib/_weakrefset.py:37: 60.0 KiB
    self.data = set()
#10: Lib/base64.py:142: 59.8 KiB
    _b32tab2 = [a + b for a in _b32tab for b in _b32tab]
6220 other: 3602.8 KiB
Total allocated size: 5303.1 KiB

See for more options.

27.7.2. API

tracemalloc.clear_traces()

Clear traces of memory blocks allocated by Python.

27.7.2.2. 域过滤器

Filter traces of memory blocks by their address space (domain).

3.6 新版功能.

inclusive

If inclusive is True (include), match memory blocks allocated in the address space domain.

If inclusive is False (exclude), match memory blocks not allocated in the address space .
domain

Address space of a memory block (int). Read-only property.

27.7.2.3. 过滤器

class tracemalloc.Filter(inclusive: bool, filename_pattern: str, lineno: int=None, all_frames: bool=False, domain: int=None)

对内存块的跟踪进行筛选。

See the function for the syntax of filename_pattern. The '.pyc' file extension is replaced with '.py'.

示例：

Filter(True, subprocess.__file__) only includes traces of the subprocess module
Filter(False, tracemalloc.__file__) excludes traces of the module
excludes empty tracebacks

在 3.5 版更改: The '.pyo' file extension is no longer replaced with '.py'.

在 3.6 版更改: Added the domain attribute.

domain

Address space of a memory block (int or None).
inclusive

If inclusive is True (include), only match memory blocks allocated in a file with a name matching at line number lineno.

If inclusive is False (exclude), ignore memory blocks allocated in a file with a name matching at line number lineno.
lineno

Line number (int) of the filter. If lineno is None, the filter matches any line number.
filename_pattern

Filename pattern of the filter (str). Read-only property.
all_frames

If all_frames is True, all frames of the traceback are checked. If all_frames is False, only the most recent frame is checked.

This attribute has no effect if the traceback limit is 1. See the function and Snapshot.traceback_limit attribute.

27.7.2.4. Frame

class tracemalloc.Frame

Frame of a traceback.

The Traceback class is a sequence of instances.

filename

文件名(str).
lineno

行号 (int).

class tracemalloc.Snapshot

Snapshot of traces of memory blocks allocated by Python.

The take_snapshot() function creates a snapshot instance.

compare_to(old_snapshot: Snapshot, key_type: str, cumulative: bool=False)

Compute the differences with an old snapshot. Get statistics as a sorted list of instances grouped by key_type.

See the Snapshot.statistics() method for key_type and cumulative parameters.

The result is sorted from the biggest to the smallest by: absolute value of , StatisticDiff.size, absolute value of , Statistic.count and then by .
dump(filename)

将快照写入文件

使用 load() 重载快照。
filter_traces(filters)

Create a new instance with a filtered traces sequence, filters is a list of and Filter instances. If filters is an empty list, return a new instance with a copy of the traces.

All inclusive filters are applied at once, a trace is ignored if no inclusive filters match it. A trace is ignored if at least one exclusive filter matches it.

在 3.6 版更改: DomainFilter instances are now also accepted in filters.
classmethod load(filename)

从文件载入快照。

另见 .
statistics(key_type: str, cumulative: bool=False)

获取 Statistic 信息列表，按 key_type 分组排序：

If cumulative is True, cumulate size and count of memory blocks of all frames of the traceback of a trace, not only the most recent frame. The cumulative mode can only be used with key_type equals to 'filename' and 'lineno'.

The result is sorted from the biggest to the smallest by: , Statistic.count and then by .
traceback_limit

Maximum number of frames stored in the traceback of traces: result of the when the snapshot was taken.
traces

Traces of all memory blocks allocated by Python: sequence of Trace instances.

The sequence has an undefined order. Use the method to get a sorted list of statistics.

27.7.2.6. 统计

class tracemalloc.Statistic

统计内存分配

返回 Statistic 实例的列表。.

参见类。

count

内存块数（整形）。
size

Total size of memory blocks in bytes (int).
traceback

Traceback where the memory block was allocated, Traceback instance.

27.7.2.7. StatisticDiff

class tracemalloc.StatisticDiff

Statistic difference on memory allocations between an old and a new Snapshot instance.

returns a list of StatisticDiff instances. See also the class.

count

Number of memory blocks in the new snapshot (int): 0 if the memory blocks have been released in the new snapshot.
count_diff

Difference of number of memory blocks between the old and the new snapshots (int): 0 if the memory blocks have been allocated in the new snapshot.
size

Total size of memory blocks in bytes in the new snapshot (int): 0 if the memory blocks have been released in the new snapshot.
size_diff

Difference of total size of memory blocks in bytes between the old and the new snapshots (int): 0 if the memory blocks have been allocated in the new snapshot.
traceback

Traceback where the memory blocks were allocated, Traceback instance.

27.7.2.8. 跟踪

class tracemalloc.Trace

Trace of a memory block.

The Snapshot.traces attribute is a sequence of instances.

size

Size of the memory block in bytes (int).
traceback

Traceback where the memory block was allocated, Traceback instance.

class tracemalloc.Traceback

Sequence of instances sorted from the most recent frame to the oldest frame.

A traceback contains at least 1 frame. If the tracemalloc module failed to get a frame, the filename "<unknown>" at line number 0 is used.

When a snapshot is taken, tracebacks of traces are limited to get_traceback_limit() frames. See the function.

The Trace.traceback attribute is an instance of instance.

format(limit=None)

Format the traceback as a list of lines with newlines. Use the linecache module to retrieve lines from the source code. If limit is set, only format the limit most recent frames.

Similar to the function, except that format() does not include newlines.

示例:
```
print("Traceback (most recent call first):")
for line in traceback:
```
输出: