Object Pooling for Polygon Vertex Buffers

Every containment test that materializes a fresh coordinate array pays the CPython allocator, and at 25k evaluations/sec that per-call allocation is what feeds the generational garbage collector until it stops the world for a double-digit-millisecond pause squarely on the hot path. This page expands the object-tax model from Memory Footprint of Streaming Polygon Indexes and the broader spatial index lookup architecture, narrowing to one lever: reusing a fixed set of numpy vertex buffers from a freelist so the steady-state allocation rate on the evaluation path drops toward zero, and the collector has nothing to sweep.

Concept and specification #

An object pool trades peak memory for allocation rate. Instead of allocating a (V, 2) float64 buffer per evaluation and letting it die, you preallocate $P$ buffers once, hand them out on checkout, and return them on release. If the pool is sized to the concurrency ceiling, the steady-state allocation rate on the hot path is zero.

Let $r$ be evaluations per second, $b$ the bytes per vertex buffer, and $h$ the pool hit ratio (fraction of checkouts served from the freelist). The allocated bytes per second is

$$A=r\cdot b\cdot (1-h)$$

A naive path has $h=0$ and pays the full $A=r\cdot b$. A correctly sized pool drives $h\to 1$, so $A\to 0$ regardless of $r$. For $r=25,000$/s and a 120-vertex buffer at $b=120\times 2\times 8=1920$ bytes, the naive path churns ~46 MB/s of transient arrays; at $h=0.98$ the pool leaves under 1 MB/s. GC pause frequency scales with gen-0 promotions, which scale with $A$, so collapsing $A$ collapses the tail.

The buffer is fixed-capacity and reused with a live-length view, so a 40-vertex polygon and a 120-vertex polygon share the same backing store; only the [:n] slice differs. Pairing the pool with __slots__ on the wrapper removes the per-instance __dict__, and setting PYTHONMALLOC=pymalloc_debug off in production keeps pymalloc’s arena reuse intact for the small wrapper objects.

Step-by-step implementation #

Prerequisites: Python 3.11+, numpy>=1.26. Input is a stream of polygon vertex rings; output is a reusable float64 buffer plus a live length. The pool is thread-safe via a single lock; for asyncio it is contention-free because the event loop is single-threaded.

1. Define a fixed-capacity buffer wrapper with __slots__. No __dict__, so each wrapper is a few dozen bytes and never triggers dict resizing.

import numpy as np

class VertexBuffer:
    __slots__ = ("_backing", "n")                     # no per-instance dict

    def __init__(self, capacity: int) -> None:
        self._backing: np.ndarray = np.empty((capacity, 2), dtype=np.float64)
        self.n: int = 0                               # live vertex count

    @property
    def view(self) -> np.ndarray:
        return self._backing[: self.n]                # zero-copy slice of live rows

    def load(self, ring: np.ndarray) -> None:
        n = ring.shape[0]
        if n > self._backing.shape[0]:                # oversized polygon: caller must fallback
            raise ValueError("ring exceeds buffer capacity")
        self._backing[:n] = ring                      # copy into reused store, no alloc
        self.n = n

2. Build the freelist pool. Checkout pops a preallocated buffer; release pushes it back after zeroing the length so stale rows can never leak into the next evaluation.

import threading
from collections import deque

class BufferPool:
    __slots__ = ("_free", "_capacity", "_lock", "hits", "misses")

    def __init__(self, size: int, capacity: int) -> None:
        self._capacity = capacity
        self._free: deque[VertexBuffer] = deque(
            VertexBuffer(capacity) for _ in range(size)   # preallocate once
        )
        self._lock = threading.Lock()
        self.hits: int = 0
        self.misses: int = 0

    def checkout(self) -> VertexBuffer:
        with self._lock:
            if self._free:
                self.hits += 1
                return self._free.popleft()
        self.misses += 1
        return VertexBuffer(self._capacity)          # transient fallback on exhaustion

    def release(self, buf: VertexBuffer) -> None:
        buf.n = 0                                     # invalidate live length before reuse
        with self._lock:
            self._free.append(buf)

3. Wrap checkout/release in a context manager. This guarantees the buffer returns to the freelist even if the containment test raises.

from contextlib import contextmanager
from collections.abc import Iterator

@contextmanager
def leased(pool: BufferPool, ring: np.ndarray) -> Iterator[VertexBuffer]:
    buf = pool.checkout()
    try:
        buf.load(ring)
        yield buf
    finally:
        pool.release(buf)                            # always return, even on error

Gotcha: never let a view escape the with block. The slice aliases pooled memory; once released the buffer can be handed to another evaluation and overwrite the array under you. Copy out any result you need to keep.

4. Use it on the evaluation path. The point-in-polygon test consumes the live view directly, allocating nothing.

def evaluate(pool: BufferPool, ring: np.ndarray, pt: tuple[float, float]) -> bool:
    with leased(pool, ring) as buf:
        return _ray_cast(buf.view, pt[0], pt[1])     # operates on reused store

Benchmark and verification #

Measure allocations/sec with tracemalloc and GC pauses with gc.callbacks, running the same 25k-eval/sec workload against ~5k polygons before and after pooling.

import gc
import time

pauses_ms: list[float] = []

def _gc_probe(phase: str, info: dict) -> None:
    if phase == "start":
        _gc_probe.t = time.perf_counter()            # type: ignore[attr-defined]
    elif phase == "stop":
        pauses_ms.append((time.perf_counter() - _gc_probe.t) * 1e3)  # type: ignore

gc.callbacks.append(_gc_probe)                       # records every collection's wall time

Representative figures from a 10-minute soak on a single core:

The 96% cut in allocations/sec is the direct cause of the pause collapse; the small RSS increase is the preallocated pool paid up front — the deliberate memory-for-latency trade. Verify $h\ge 0.98$ in steady state; a lower ratio means the pool is undersized and misses are silently re-introducing churn.

Failure modes and edge cases #

• Pool leak. A code path that checks out without releasing drains the freelist; every subsequent checkout falls through to the transient fallback and $h$ collapses to zero, quietly restoring the original allocation storm. Always lease through the context manager and alert if misses grows unbounded.
• Buffer aliasing bug. Returning a view that outlives the lease lets a later evaluation overwrite live data, producing wrong containment answers with no crash. Copy before you release, and never store a view in a cache — the failure is silent and correctness-affecting.
• Oversized polygons. A ring larger than $V_{\max }$ raises on load; handle it with a transient full-size allocation rather than growing every pooled buffer to the worst case, which would waste RSS on the common small polygon. Size $V_{\max }$ at the 99th percentile vertex count.
• Stale length reuse. Forgetting to zero n on release leaks previous vertices into the next evaluation’s view. The release method resets n = 0 for exactly this reason; keep it there.
• GC still runs on wrappers. __slots__ removes the dict but the wrapper objects themselves are still tracked. Call gc.freeze() after warming the pool so the long-lived buffers move out of the gen-0/1 scan set entirely — the reclamation discipline covered in Memory Footprint of Streaming Polygon Indexes.

Related #

• Memory Footprint of Streaming Polygon Indexes — parent overview of the object tax and array-backed storage this pool implements.
• Measuring RSS Growth in Long-Running Spatial Indexes — the monitor that confirms a pool leak versus arena fragmentation.
• Memory-Constrained Spatial Processing — the latency budget the pause reduction protects.