Measure Python Performance Before You Change the Code

The slowest Python optimization I have seen was also the most confident.

A batch job took about forty minutes. The first theory was that Python loops were the problem. The second theory was that async would help. The third theory was that a native extension was inevitable. The actual bottleneck was an item in huge_list check inside a nested loop. Converting one collection to a set changed the job more than any runtime flag would have.

No cleverness. Just measurement, then the right data structure.

Python performance work goes wrong when teams skip the boring questions:

• What is slow?
• How slow is it?
• Is the bottleneck CPU, I/O, memory, allocation, serialization, database time, network time, lock contention, or import time?
• How will we know the change helped?
• What readability or operational cost are we accepting?

The Python 3.13+ era gives us more performance surfaces: a faster interpreter baseline, experimental JIT work, free-threaded builds, subinterpreters, better profiling paths, and eventually the Python 3.15 profiling package. More knobs make measurement more important, not less.

Start With the Shape of the Work#

Before touching code, classify the workload.

That table saves time because it blocks generic fixes. Async does not help a CPU-bound loop. A JIT does not help a request handler waiting on a database. Multiprocessing does not help a memory-bound pipeline if serialization dominates. A vectorized library may not help a control-heavy parser.

The first answer should be a measurement plan, not an implementation plan.

Build a Tiny Harness#

When the suspected bottleneck is local and deterministic, timeit is still a good first move.

from random import randrange
from timeit import timeit
 
 
items = [randrange(1_000_000) for _ in range(200_000)]
needles = [randrange(1_000_000) for _ in range(5_000)]
item_set = set(items)
 
 
def list_membership() -> int:
    return sum(1 for needle in needles if needle in items)
 
 
def set_membership() -> int:
    return sum(1 for needle in needles if needle in item_set)
 
 
print(timeit(list_membership, number=10))
print(timeit(set_membership, number=10))

This is not a production benchmark. It is a quick sanity check. It tells you whether the idea is worth deeper measurement.

For service code, I want stage timings instead:

from contextlib import contextmanager
from time import perf_counter
 
 
@contextmanager
def timed(label: str):
    started = perf_counter()
    try:
        yield
    finally:
        elapsed_ms = (perf_counter() - started) * 1000
        print(f"{label}={elapsed_ms:.1f}ms")
 
 
def run_batch() -> None:
    with timed("read"):
        records = read_records()
    with timed("transform"):
        transformed = transform_records(records)
    with timed("write"):
        write_records(transformed)

Ugly? A little. Useful? Very. Once the slow stage is known, replace prints with structured metrics or spans.

Use Profilers for Different Questions#

The standard profile and cProfile documentation remains useful for deterministic call profiling. Deterministic profilers answer questions like "which functions were called, how often, and how much cumulative time did they take?" They are excellent for targeted local analysis. They add overhead and can distort highly concurrent or latency-sensitive programs.

Sampling profilers answer a different question: "where was the program spending time when sampled?" They are often better for production-like systems because they impose less overhead and can observe long-running processes.

Python 3.15 is especially interesting here. The draft 3.15 release notes describe a new profiling package from PEP 799, with deterministic tracing and a statistical sampling profiler named Tachyon. The profiling.sampling documentation describes attach-by-PID and output formats such as flame graphs.

Because 3.15 is prerelease on June 16, 2026, I would treat that as a pilot path, not the baseline for production services. In current production, tools such as py-spy, Scalene, Memray, and cProfile still earn their keep depending on the question.

Data Structures Beat Micro-Optimizations#

Most Python performance wins I trust start with data structure choice.

if customer_id in active_customer_ids:
    ...

The performance depends on the type of active_customer_ids.

The original batch job did not need a new runtime. It needed a set.

The tradeoff is memory and semantics. Sets use more memory than lists and discard order. Dicts are great for lookup but can make ordering assumptions less obvious. Deques are excellent queues but not list replacements. Choose the structure that matches the operation you perform most.

Generators Are for Streaming, Not Virtue#

Generators reduce memory when the pipeline can stream.

def transformed_rows(rows: Iterable[RawRow]) -> Iterator[CleanRow]:
    for row in rows:
        yield clean_row(row)

They are the wrong choice when the next step needs the whole collection repeatedly, random access, or a length. A list comprehension can be faster and clearer when materialization is intentional.

clean_rows = [clean_row(row) for row in rows]

The performance standard is not "generators good, lists bad." It is "do we need the collection or the stream?"

Vectorize Numeric and Columnar Work#

For numerical and tabular workloads, the fastest Python code is often the code that stops looping in Python.

NumPy remains foundational for array math. Polars is a strong fit for columnar dataframe work, especially lazy query plans and multi-threaded execution. PyArrow is the interchange and columnar memory layer that often sits underneath serious data systems.

The mistake is vectorizing blindly.

If the workload is a straightforward column transform over millions of rows, vectorization is usually the right direction. If the workload is branch-heavy, row-specific, external-API-driven, or requires complex Python object behavior, vectorization may make the code harder to read without moving the bottleneck.

Measure the stage, then choose the library.

Treat the JIT as Experimental#

Python 3.13 introduced an experimental JIT. The 3.13 release notes describe it as disabled by default with modest expected improvements. PEP 744 is even clearer: until the JIT is no longer experimental, it should not be used in production and may be broken or removed without warning.

That does not mean ignore it.

It means benchmark it in a controlled lane:

python benchmark_pipeline.py
PYTHON_JIT=1 python benchmark_pipeline.py

Record runtime, memory, startup behavior, and variance. If the workload is long-running and CPU-bound with tight loops, the result may be interesting. If the workload is I/O-heavy, import-heavy, or dominated by native libraries already doing the work, the JIT may not matter.

Free-Threading Is a Concurrency Decision#

Free-threaded Python is performance-relevant, but not as a magic speed switch. It changes the concurrency model. PEP 779 makes free-threaded Python officially supported but still optional in 3.14. The free-threading HOWTO emphasizes dependency compatibility and runtime checks such as sys._is_gil_enabled().

If the workload is CPU-bound and thread-parallel, it may be a real option. If the service is I/O-bound, async and better backpressure may still be the right answer. If dependencies silently re-enable the GIL or are not thread-safe, the pilot is not ready.

A Performance Review Checklist#

I want this checklist before approving a performance PR:

1. Symptom
   - What user, job, or system behavior is too slow?
   - What is the current p50, p95, p99, throughput, or runtime?
 
2. Measurement
   - Which profiler, trace, benchmark, or metric identified the bottleneck?
   - Can another engineer reproduce the measurement?
 
3. Bottleneck class
   - CPU, I/O, memory, allocation, serialization, import, lock, database, network?
 
4. Change
   - Algorithm or data structure?
   - Vectorized/native library?
   - Concurrency model?
   - Runtime flag or interpreter version?
 
5. Tradeoff
   - More memory?
   - Less readability?
   - New dependency?
   - Harder debugging?
   - Different failure mode?
 
6. Result
   - Before and after numbers.
   - Same input data.
   - Same hardware or clearly documented environment.
   - Variance reported, not one lucky run.

Performance work should leave a trail. The next person should know why the code is less obvious, why the dependency exists, or why the runtime flag is set.

The best Python optimization is usually not heroic. It is the small change made after the measurement told you where to look.