Lazy import only required plugin: `open` 2.3-15.6x & `save` 2.2-9x faster #9398

hugovk · 2026-01-18T23:14:35Z

preinit + init

When loading or saveing an image, if Pillow isn't yet initialised, we call a preinit function.

This loads five drivers for five popular formats by importing their plugins: BMP, GIF, JPEG, PPM and PPM.

Then we check each of these plugins in turn to see if one will accept it (which usually involves reading at least part of the image data for a magic prefix), and if so, then load it.

If none of these common five match, we call init, which imports the remaining 42 plugins. We then check each of these for a match.

This has been the case since at least PIL 1.1.1 (released in 2000).

Lazy

This is all a bit wasteful if we only need one or two image formats during a program's lifetime. (Longer running ones may need a few more, but unlikely all 47, and in any case speed is less of a worry for long-running programs.)

This PR adds a mapping of common extensions to plugins. Before preinit and init, we can do a very cheap lookup, and may save us importing many plugins, and save us trying to see load with many plugins.

Of course, we may have an image without an extension, or with the "wrong" extension, but that's fine, I expect it's rare and anyway we'll fallback to the full preinit -> init flow.

Benchmarks

Combined

This scripts times the new code:

Opening a PNG, and compares it with explicitly calling preinit first.
Opening a WEBP, and compares it with explicitly calling init first.
Saving a PNG, and compares it with explicitly calling preinit first.
Saving a WEBP, and compares it with explicitly calling init first.

import statistics
import subprocess
import sys

OPEN_TEMPLATE = """
import time
from PIL import Image
start = time.perf_counter()
{preload}
im = Image.open("Tests/images/hopper.{ext}")
print(f"{{(time.perf_counter() - start)*1000:.2f}}")
"""

SAVE_TEMPLATE = """
import time
import tempfile
from PIL import Image
im = Image.new("RGB", (100, 100))
start = time.perf_counter()
{preload}
with tempfile.NamedTemporaryFile(suffix=".{ext}") as f:
    im.save(f.name)
print(f"{{(time.perf_counter() - start)*1000:.2f}}")
"""


def run_benchmark(code: str, iterations: int = 20) -> list[float]:
    times = []
    for _ in range(iterations):
        result = subprocess.run(
            [sys.executable, "-c", code], capture_output=True, text=True
        )
        times.append(float(result.stdout.strip()))
    return times


def print_results(name: str, times: list[float]) -> None:
    print(f"  {name}:\tmedian {statistics.median(times):.2f}ms, min {min(times):.2f}ms")


def main():
    print("Benchmarking...\n")

    open_png_opt = run_benchmark(OPEN_TEMPLATE.format(ext="png", preload=""))
    open_png_preinit = run_benchmark(OPEN_TEMPLATE.format(ext="png", preload="Image.preinit()"))
    open_webp_opt = run_benchmark(OPEN_TEMPLATE.format(ext="webp", preload=""))
    open_webp_init = run_benchmark(OPEN_TEMPLATE.format(ext="webp", preload="Image.init()"))

    save_png_opt = run_benchmark(SAVE_TEMPLATE.format(ext="png", preload=""))
    save_png_preinit = run_benchmark(SAVE_TEMPLATE.format(ext="png", preload="Image.preinit()"))
    save_webp_opt = run_benchmark(SAVE_TEMPLATE.format(ext="webp", preload=""))
    save_webp_init = run_benchmark(SAVE_TEMPLATE.format(ext="webp", preload="Image.init()"))

    print("Open:")
    print_results("PNG lazy    (1 plugin)", open_png_opt)
    print_results("PNG preinit (5 plugins)", open_png_preinit)
    print_results("WebP lazy   (1 plugin)", open_webp_opt)
    print_results("WebP init   (47 plugins)", open_webp_init)

    print("\nSave:")
    print_results("PNG lazy    (1 plugin)", save_png_opt)
    print_results("PNG preinit (5 plugins)", save_png_preinit)
    print_results("WebP lazy   (1 plugin)", save_webp_opt)
    print_results("WebP init   (47 plugins)", save_webp_init)

    print("\nSpeedup (lazy vs preload):")
    print(f"  Open PNG:  {statistics.median(open_png_preinit) / statistics.median(open_png_opt):.1f}x")
    print(f"  Open WebP: {statistics.median(open_webp_init) / statistics.median(open_webp_opt):.1f}x")
    print(f"  Save PNG:  {statistics.median(save_png_preinit) / statistics.median(save_png_opt):.1f}x")
    print(f"  Save WebP: {statistics.median(save_webp_init) / statistics.median(save_webp_opt):.1f}x")


if __name__ == "__main__":
    main()

Python 3.10

Open:
  PNG lazy    (1 plugin):       median 3.17ms, min 3.14ms
  PNG preinit (5 plugins):      median 7.20ms, min 7.04ms
  WebP lazy   (1 plugin):       median 1.35ms, min 1.32ms
  WebP init   (47 plugins):     median 21.05ms, min 20.88ms

Save:
  PNG lazy    (1 plugin):       median 3.76ms, min 3.70ms
  PNG preinit (5 plugins):      median 8.44ms, min 7.62ms
  WebP lazy   (1 plugin):       median 2.49ms, min 2.29ms
  WebP init   (47 plugins):     median 22.42ms, min 22.00ms

Speedup (lazy vs preload):
  Open PNG:  2.3x
  Open WebP: 15.6x
  Save PNG:  2.2x
  Save WebP: 9.0x

Python 3.14

Open:
  PNG lazy    (1 plugin):       median 3.13ms, min 3.04ms
  PNG preinit (5 plugins):      median 8.27ms, min 7.98ms
  WebP lazy   (1 plugin):       median 1.56ms, min 1.47ms
  WebP init   (47 plugins):     median 21.98ms, min 20.89ms

Save:
  PNG lazy    (1 plugin):       median 4.51ms, min 3.87ms
  PNG preinit (5 plugins):      median 9.72ms, min 9.06ms
  WebP lazy   (1 plugin):       median 2.83ms, min 2.34ms
  WebP init   (47 plugins):     median 22.41ms, min 21.65ms

Speedup (lazy vs preload):
  Open PNG:  2.6x
  Open WebP: 14.0x
  Save PNG:  2.2x
  Save WebP: 7.9x

Read

These are hyperfine comparisons between main and the new code, and include the overhead of the Python interpreter startup and PIL import Image etc.

import sys
from PIL import Image

im = Image.open(f"Tests/images/hopper.{sys.argv[1]}")

Read png (preinit group)

❯ hyperfine --warmup 5 \
--prepare "git checkout main" "python3 1.py png # main" \
--prepare "git checkout lazy" "python3 1.py png # lazy"
Benchmark 1: python3 1.py png # main
  Time (mean ± σ):      58.9 ms ±   1.4 ms    [User: 48.5 ms, System: 8.8 ms]
  Range (min … max):    57.2 ms …  63.1 ms    28 runs

Benchmark 2: python3 1.py png # lazy
  Time (mean ± σ):      55.4 ms ±   4.6 ms    [User: 44.8 ms, System: 8.4 ms]
  Range (min … max):    52.0 ms …  70.8 ms    29 runs

Summary
  python3 1.py png # lazy ran
    1.06 ± 0.09 times faster than python3 1.py png # main

Read webp (init group)

❯ hyperfine --warmup 5 \
--prepare "git checkout main" "python3 1.py webp # main" \
--prepare "git checkout lazy" "python3 1.py webp # lazy"
Benchmark 1: python3 1.py webp # main
  Time (mean ± σ):      74.0 ms ±   2.8 ms    [User: 61.3 ms, System: 10.9 ms]
  Range (min … max):    71.1 ms …  82.1 ms    24 runs

Benchmark 2: python3 1.py webp # lazy
  Time (mean ± σ):      55.6 ms ±   5.7 ms    [User: 44.1 ms, System: 9.1 ms]
  Range (min … max):    50.8 ms …  70.6 ms    26 runs

Summary
  python3 1.py webp # lazy ran
    1.33 ± 0.15 times faster than python3 1.py webp # main

Save

import sys
from PIL import Image

im = Image.new("RGB", (100, 100), "red")
im.save(f"/tmp/out.{sys.argv[1]}")

Save png (preinit group)

❯ hyperfine --warmup 5 \
--prepare "git checkout main" "python3 2.py png # main" \
--prepare "git checkout lazy" "python3 2.py png # lazy"
Benchmark 1: python3 2.py png # main
  Time (mean ± σ):      68.3 ms ±  12.7 ms    [User: 54.5 ms, System: 10.5 ms]
  Range (min … max):    58.6 ms … 109.1 ms    26 runs

Benchmark 2: python3 2.py png # lazy
  Time (mean ± σ):      60.2 ms ±   7.5 ms    [User: 47.3 ms, System: 9.5 ms]
  Range (min … max):    53.5 ms …  80.1 ms    28 runs

Summary
  python3 2.py png # lazy ran
    1.13 ± 0.25 times faster than python3 2.py png # main

Save webp (init group)

❯ hyperfine --warmup 5 \
--prepare "git checkout main" "python3 2.py webp # main" \
--prepare "git checkout lazy" "python3 2.py webp # lazy"
Benchmark 1: python3 2.py webp # main
  Time (mean ± σ):      75.6 ms ±   2.2 ms    [User: 62.4 ms, System: 11.3 ms]
  Range (min … max):    72.4 ms …  80.8 ms    24 runs

Benchmark 2: python3 2.py webp # lazy
  Time (mean ± σ):      57.6 ms ±   7.4 ms    [User: 44.7 ms, System: 9.2 ms]
  Range (min … max):    51.9 ms …  79.3 ms    23 runs

Summary
  python3 2.py webp # lazy ran
    1.31 ± 0.17 times faster than python3 2.py webp # main

src/PIL/Image.py

…urn early Co-authored-by: Andrew Murray <[email protected]>

hugovk · 2026-01-19T09:50:54Z

src/PIL/Image.py

+}
+
+
+def _load_plugin_for_extension(ext: str | bytes) -> bool:


Also considered calling this _lazy_init to parallel preinit and init, but I don't mind :)

Suggested change

def _load_plugin_for_extension(ext: str | bytes) -> bool:

def _lazy_init(ext: str | bytes) -> bool:

Yeah, could go either way. I think _load_plugin_for_extension is clearer about its purpose, but _lazy_init highlights the relationship to init and preinit

hugovk · 2026-01-19T09:51:11Z

src/PIL/Image.py

        ext = filename_ext.decode() if isinstance(filename_ext, bytes) else filename_ext

+        # Try loading only the plugin for this extension first
+        if not _load_plugin_for_extension(ext):


Suggested change

if not _load_plugin_for_extension(ext):

if not _lazy_init(ext):

hugovk · 2026-01-19T09:51:18Z

src/PIL/Image.py

+    # Try to load just the plugin needed for this file extension
+    # before falling back to preinit() which loads common plugins
+    ext = os.path.splitext(filename)[1] if filename else ""
+    if not (ext and _load_plugin_for_extension(ext)):


Suggested change

if not (ext and _load_plugin_for_extension(ext)):

if not (ext and _lazy_init(ext)):

Lazy import only required plugin

6b9de40

hugovk added the Performance label Jan 18, 2026

radarhere reviewed Jan 19, 2026

View reviewed changes

src/PIL/Image.py Show resolved Hide resolved

radarhere mentioned this pull request Jan 19, 2026

Check that _EXTENSION_PLUGIN contains all registered extensions hugovk/Pillow#143

Merged

radarhere and others added 5 commits January 19, 2026 17:13

Check that _EXTENSION_PLUGIN contains all registered extensions

1baf141

Cleanup .spider extension registered by test code during save

9c8059f

Do not register empty extension

b06118c

Check that _EXTENSION_PLUGIN contains all registered extensions (#143)

7f38f98

If plugin has already been imported and registered the extension, ret…

096c479

…urn early Co-authored-by: Andrew Murray <[email protected]>

hugovk commented Jan 19, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Lazy import only required plugin: `open` 2.3-15.6x & `save` 2.2-9x faster #9398

Lazy import only required plugin: `open` 2.3-15.6x & `save` 2.2-9x faster #9398

hugovk commented Jan 18, 2026

Uh oh!

Uh oh!

hugovk Jan 19, 2026

Uh oh!

radarhere Jan 19, 2026

Uh oh!

hugovk Jan 19, 2026

Uh oh!

hugovk Jan 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	def _load_plugin_for_extension(ext: str \| bytes) -> bool:
	def _lazy_init(ext: str \| bytes) -> bool:

	if not _load_plugin_for_extension(ext):
	if not _lazy_init(ext):

	if not (ext and _load_plugin_for_extension(ext)):
	if not (ext and _lazy_init(ext)):

Uh oh!

Lazy import only required plugin: open 2.3-15.6x & save 2.2-9x faster #9398

Are you sure you want to change the base?

Lazy import only required plugin: open 2.3-15.6x & save 2.2-9x faster #9398

Conversation

hugovk commented Jan 18, 2026

preinit + init

Lazy

Benchmarks

Combined

Python 3.10

Python 3.14

Read

Read png (preinit group)

Read webp (init group)

Save

Save png (preinit group)

Save webp (init group)

Uh oh!

Uh oh!

hugovk Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

radarhere Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

hugovk Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

hugovk Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Lazy import only required plugin: `open` 2.3-15.6x & `save` 2.2-9x faster #9398

Lazy import only required plugin: `open` 2.3-15.6x & `save` 2.2-9x faster #9398