Skip to content

Commit 2843bae

Browse files
committed
PYTHON-4099 Add contributing docs for memory profiling
1 parent 49e59d4 commit 2843bae

File tree

1 file changed

+23
-1
lines changed

1 file changed

+23
-1
lines changed

CONTRIBUTING.md

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -524,8 +524,10 @@ Use this generated file as a starting point for the completed conversion.
524524
525525
The script is used like so: `python tools/convert_test_to_async.py [test_file.py]`
526526
527-
## Generating a flame graph using py-spy
527+
## CPU profiling
528+
528529
To profile a test script and generate a flame graph, follow these steps:
530+
529531
1. Install `py-spy` if you haven't already:
530532
```bash
531533
pip install py-spy
@@ -535,6 +537,26 @@ To profile a test script and generate a flame graph, follow these steps:
535537
(Note: on macOS you will need to run this command using `sudo` to allow `py-spy` to attach to the Python process.)
536538
4. If you need to include native code (for example the C extensions), profiling should be done on a Linux system, as macOS and Windows do not support the `--native` option of `py-spy`.
537539
Creating an ubuntu Evergreen spawn host and using `scp` to copy the flamegraph `.svg` file back to your local machine is the best way to do this.
540+
5. You can then view the flamegraph using an SVG viewer like a browser.
541+
542+
## Memory profiling
543+
544+
To test for a memory leak or any memory-related issues, the current best tool is [memray](https://bloomberg.github.io/memray/overview.html).
545+
In order to include code from our C extensions, it must be run in native mode, on Linux.
546+
To do so, either spin up an Ubuntu docker container or an Ubuntu Evergreen spawn host.
547+
548+
From the spawn host or Ubuntu image, do the following:
549+
550+
1. Install `memray` if you haven't already:
551+
```bash
552+
pip install memray
553+
```
554+
2. Inside your test script, perform any required setup and then loop over the code you want to profile for improved sampling.
555+
3. Run memray with the script under test with the `--native` flag, e.g. `python -m memray run --native -o test.bin <path/to/script>`.
556+
4. Generate the flamegraph with `python -m memray flamegraph -o test.html test.bin`.
557+
See the [docs](https://bloomberg.github.io/memray/flamegraph.html) for more options.
558+
5. Then, from the host computer, use either scp or docker cp to copy the flamegraph, e.g. `scp [email protected]:/home/ubuntu/test.html .`.
559+
6. You can then view the flamegraph html in a browser.
538560
539561
## Dependabot updates
540562

0 commit comments

Comments
 (0)