ProcessPoolExecutor vs Numba JIT for Spatial Math

Point-in-polygon evaluation is CPU-bound, and under the GIL a single Python thread serializes it onto one core no matter how many the box has. There are two production-grade escapes, and they occupy opposite ends of the overhead spectrum: ProcessPoolExecutor spreads work across separate interpreter processes that sidestep the GIL by not sharing one, at the cost of inter-process communication; and Numba’s @njit(nogil=True) compiles the kernel to machine code that releases the GIL in-process, so ordinary threads run it in parallel with near-zero per-call overhead. This page expands the offload-boundary problem from Async Python Execution Patterns for Spatial Math under the wider Core Architecture & Latency Constraints model, and pairs with the runnable measurements in benchmarking spatial containment in async Python. The question it answers by name: for a hot containment kernel, does multiprocessing or a nogil JIT give more throughput per core once you account for the overhead each imposes?

Concept and specification #

Both approaches turn one saturated core into $P$ busy cores, but they pay for it differently.

ProcessPoolExecutor forks or spawns $P$ worker processes, each a full Python interpreter with its own GIL. A task’s arguments are pickled, pushed through a pipe, unpickled in the worker, and the result travels back the same way. That round trip is the fixed cost — roughly 50 µs per task once you eliminate array copies by putting the coordinate buffer in multiprocessing.shared_memory and passing only the block name. The model wins when each task carries enough compute to bury the IPC cost.

Numba @njit(nogil=True) compiles the kernel to native code via LLVM on first call. Because the compiled function holds no Python objects, it can drop the GIL for its entire duration, so a ThreadPoolExecutor of ordinary threads executes it truly in parallel inside one process. There is no serialization and no process boundary: per-call overhead is the ~0.2 µs of a function dispatch after the one-time compile.

Model the effective throughput of a batch of $B$ points per task across $P$ workers, with kernel cost $c_k$ per point and IPC cost $c_{\text{ipc}}$ per task:

$${\Theta }_{\text{proc}}=\frac{P\cdot B}{B\cdot c_k+c_{\text{ipc}}},{\Theta }_{\text{numba}}\approx \frac{P}{c_k}$$

The process pool only approaches $P/c_k$ as $B\to \infty$; the fraction wasted on IPC is $c_{\text{ipc}}/(B{c}_k)$. With $c_{\text{ipc}}=50,\mu s$ and $c_k=0.2,\mu s$, a task must carry $B\gg 250$ points before the overhead falls below 20%, and roughly $B=5000$ before it drops under 5%. Numba pays no such tax at any batch size — but it pays a fixed compile cost up front.

Step-by-step implementation #

Prerequisites: Python 3.11+, numpy>=1.26, numba>=0.59. The kernel is a ray-casting point-in-polygon test; input is a (N, 2) float64 coordinate array and a polygon’s separated vertex arrays.

1. Write the Numba kernel with the GIL released. Pass only raw NumPy scalars and arrays — no Python objects — so the compiler can drop the GIL.

import numpy as np
from numba import njit


@njit(nogil=True, cache=True, fastmath=True)
def _point_in_polygon(px: float, py: float, vx: np.ndarray, vy: np.ndarray) -> bool:
    """Ray-cast containment; nogil so threads run it truly in parallel."""
    inside = False
    n = vx.shape[0]
    j = n - 1
    for i in range(n):
        # Edge straddles the horizontal ray and the crossing is to the right of px.
        if ((vy[i] > py) != (vy[j] > py)) and (
            px < (vx[j] - vx[i]) * (py - vy[i]) / (vy[j] - vy[i]) + vx[i]
        ):
            inside = not inside
        j = i
    return inside


@njit(nogil=True, cache=True)
def contains_batch(
    pxs: np.ndarray, pys: np.ndarray, vx: np.ndarray, vy: np.ndarray, out: np.ndarray
) -> None:
    for k in range(pxs.shape[0]):
        out[k] = _point_in_polygon(pxs[k], pys[k], vx, vy)

2. Pre-warm the JIT at startup so the first request never pays the compile. Call the kernel once on dummy data during boot; cache=True persists the compiled artifact to __pycache__ across restarts.

def warm_up() -> None:
    dummy = np.zeros(1, dtype=np.float64)
    out = np.zeros(1, dtype=np.bool_)
    contains_batch(dummy, dummy, dummy, dummy, out)  # forces the ~0.8 s compile once

3. Run it in parallel with a thread pool from asyncio. Because the kernel is nogil, threads — not processes — scale it without IPC.

import asyncio
from concurrent.futures import ThreadPoolExecutor


async def numba_route(
    pxs: np.ndarray, pys: np.ndarray, vx: np.ndarray, vy: np.ndarray,
    pool: ThreadPoolExecutor,
) -> np.ndarray:
    out = np.empty(pxs.shape[0], dtype=np.bool_)
    loop = asyncio.get_running_loop()
    # No pickling, no copy: threads share the arrays and release the GIL in the kernel.
    await loop.run_in_executor(pool, contains_batch, pxs, pys, vx, vy, out)
    return out

Gotcha: use a ThreadPoolExecutor with Numba, not a ProcessPoolExecutor. Wrapping a nogil kernel in a process pool reintroduces the exact IPC cost you compiled it to avoid.

4. The ProcessPoolExecutor alternative: share the buffer, pass only its name. Pickling the coordinate array per task is the trap; put it in shared_memory and hand workers the block name and a slice range.

from concurrent.futures import ProcessPoolExecutor
from multiprocessing import shared_memory


def _worker(
    shm_name: str, shape: tuple[int, int], lo: int, hi: int,
    vx: np.ndarray, vy: np.ndarray,
) -> np.ndarray:
    shm = shared_memory.SharedMemory(name=shm_name)          # attach, no copy
    coords = np.ndarray(shape, dtype=np.float64, buffer=shm.buf)
    res = np.empty(hi - lo, dtype=np.bool_)
    for k in range(lo, hi):
        res[k - lo] = _pip(coords[k, 0], coords[k, 1], vx, vy)  # pure-Python fallback kernel
    shm.close()                                              # detach; do NOT unlink here
    return res

Benchmark and verification #

Single 8-core host, 1M points against a 64-vertex polygon, batched into tasks. Numba uses a ThreadPoolExecutor(8); the process pool uses ProcessPoolExecutor(8) with a shared_memory coordinate block.

The small-task row is the decisive one: at 100 points per task the process pool loses most of its cores to IPC while Numba’s threads are unaffected, because Numba’s overhead is independent of batch size. The process pool only becomes competitive at large batches, and even then it carries eight interpreter copies in RSS. A minimal harness to reproduce the throughput figures:

import time


def bench_throughput(route, coords, runs: int = 20) -> float:
    best = 0.0
    for _ in range(runs):
        t0 = time.perf_counter()
        route(coords)                                # closure over the executor + polygon
        rate = len(coords) / (time.perf_counter() - t0)
        best = max(best, rate)                        # points/sec at peak
    return best

Verify correctness against a reference before trusting either path: run the same batch through a plain Shapely contains and require bit-identical boolean output. Enable NUMBA_DEBUGINFO=1 and check contains_batch.inspect_types() to confirm the kernel compiled in nopython mode — a silent fallback to object mode erases the entire speedup. Watch the event loop with py-spy under load to confirm neither offload is blocking the reactor thread.

Failure modes and edge cases #

• Pickling large arrays into the process pool. Passing a NumPy array directly as a task argument pickles and copies it per task, which at 50k tasks/sec dwarfs the kernel cost and can OOM the box. Always route bulk data through shared_memory and pass only the block name and a slice; pickle just the tiny per-task metadata.
• Numba unsupported constructs. @njit rejects Python lists of Shapely geometries, dicts of ragged arrays, and most object types; feed it only scalars, NumPy arrays, and typed containers. A construct it cannot lower triggers a fallback to slow object mode (or, with nopython, a compile error) — assert on inspect_types() in CI.
• Cold-start compile stalls. The first call pays ~0.8 s of JIT. Without warm_up() at boot, the first production request stalls and trips latency alerts; and any change to the kernel source or Numba version invalidates the cache=True artifact, so a deploy silently re-incurs the compile on the first hit of each new pod.
• Fork vs spawn semantics. On Linux ProcessPoolExecutor forks by default and inherits open sockets and a warm interpreter; on macOS and Windows it spawns, re-importing your module and re-triggering module-level side effects. Use a forkserver context and guard startup code under if __name__ == "__main__": to avoid re-running warm-up per worker.
• shared_memory leaks. A SharedMemory block that is never unlink-ed survives the process and leaks until reboot, and Python’s resource_tracker emits warnings about it. Create and unlink the block from the parent only, have workers close() (never unlink), and wrap the lifecycle in a try/finally.
• GC pauses under either model. Sustained allocation of result arrays feeds generation-2 sweeps that inject tail latency; pool the output buffers and call gc.freeze() after warm-up, as covered for the surrounding pipeline in the parent async execution patterns.

Related #

• Async Python Execution Patterns for Spatial Math — the parent overview of offload boundaries and event-loop safety
• Benchmarking Spatial Containment in Async Python — the sibling with the full measurement harness these figures come from
• Core Architecture & Latency Constraints — the end-to-end latency budget these kernels fit inside
• Point-in-Polygon Algorithm Benchmarks — the ray-cast vs winding-number kernel being parallelized here