Windowed vs Continuous Geofence Evaluation Trade-offs

Once a geofence pipeline commits to streaming, one decision still remains: does it evaluate each coordinate the instant it arrives, or accumulate coordinates into a short window and evaluate them together? Continuous per-event evaluation gives the lowest possible staleness — a crossing is resolved as soon as the ping lands — but pays full Python call, index-descent, and dispatch overhead on every single event. Windowed micro-batching amortizes that overhead across many events and vectorizes the geometry, driving CPU-per-event down sharply, but every trigger inside a window is delayed by up to the window length before it can fire. This page sits under Streaming vs Batch Geofence Evaluation and the broader Core Architecture & Latency Constraints reference, and it makes the window-size decision quantitative: how large a window can you run before staleness violates the trigger deadline, and how much CPU does batching actually save?

Concept and specification #

The trade is a single axis with staleness on one end and throughput on the other. In continuous evaluation each event $i$ incurs a fixed per-event cost $c$; total CPU scales as $N\cdot c$ for $N$ events, and staleness is bounded only by processing latency. In windowed evaluation events are grouped into windows of duration $\tau$; each window pays a fixed overhead $o$ (flush, sort, batch setup) once and a reduced marginal cost $c’$ per event because the geometry vectorizes. The expected staleness a trigger sees is half the window plus its processing time:

$$\bar{S}<em>\text{window}=\frac{\tau }{2}+t</em>\text{proc},C_{\text{window}}=\frac{o}{\tau \cdot \lambda }+c’$$

where $\lambda$ is the arrival rate (events/sec), so $\tau \cdot \lambda$ is the events per window. The amortized overhead term $\frac{o}{\tau \lambda }$ shrinks as the window widens or traffic rises — batching pays off precisely when volume is high. The window must satisfy $\frac{\tau }{2}+t_{\text{proc}}\le D$ for a trigger deadline $D$, which fixes the maximum usable window:

$${\tau }_{\max }=2\cdot (D-t_{\text{proc}})$$

For a 500ms deadline and 8ms processing, ${\tau }_{\max }\approx 984$ ms, but staleness is an average of $\tau /2$, so a conservative 250ms window keeps even worst-case staleness ($\tau +t_{\text{proc}}$) comfortably inside the deadline.

Step-by-step implementation #

Prerequisites: Python 3.11+, numpy>=1.26, asyncio for the flush timer. Input is a stream of dual-timestamp telemetry as established in event-time ordering; output is committed transitions.

1. Continuous evaluator — resolve each event on arrival. Lowest staleness, scalar geometry per event.

from __future__ import annotations

async def continuous_eval(
    stream,                            # async iterator of TelemetryPoint
    route,                             # (lat, lon) -> fence_id | None
    commit,                            # (TelemetryPoint, fence_id) -> None
) -> None:
    async for p in stream:
        fence_id = route(p.lat, p.lon)  # scalar index descent + PIP per event
        commit(p, fence_id)             # staleness ≈ processing latency only

2. Windowed evaluator — accumulate for τ, then vectorize. A flush fires on time or on a size cap, and the batch resolves as one NumPy operation.

import asyncio
import time
import numpy as np

async def windowed_eval(
    stream,
    route_batch,                       # (N,2) float64 -> (N,) int32 fence ids
    commit_batch,                      # (list[TelemetryPoint], np.ndarray) -> None
    window_ms: float = 250.0,
    max_batch: int = 4096,             # size cap bounds staleness under bursts
) -> None:
    buf: list[TelemetryPoint] = []
    deadline = time.monotonic() + window_ms / 1000.0
    async for p in stream:
        buf.append(p)
        # Flush on whichever fires first: time window or size cap.
        if time.monotonic() >= deadline or len(buf) >= max_batch:
            coords = np.fromiter(
                ((q.lat, q.lon) for q in buf), dtype=np.dtype((np.float64, 2)),
                count=len(buf),
            )
            fence_ids = route_batch(coords)     # one vectorized descent
            commit_batch(buf, fence_ids)
            buf = []
            deadline = time.monotonic() + window_ms / 1000.0

Gotcha: a pure time-based flush stalls under a traffic spike — 50k events can pile into one window and blow both the batch size and staleness. The max_batch size cap is not optional; it bounds worst-case staleness to whichever of the two limits binds first.

3. Size the window against the trigger deadline. Derive ${\tau }_{\max }$ from the deadline and clamp the configured window below it.

def size_window_ms(deadline_ms: float, proc_ms: float, safety: float = 0.5) -> float:
    # tau_max = 2*(D - t_proc); apply a safety factor for tail processing.
    tau_max = 2.0 * (deadline_ms - proc_ms)
    return max(0.0, tau_max * safety)     # e.g. D=500, proc=8 -> ~246ms

Gotcha: window sizing must use the P99 of proc_ms, not the mean. Sizing against the mean leaves no headroom when a GC pause or index rebuild inflates a single flush’s processing time.

Benchmark and verification #

The change to measure is CPU-per-event and P99 staleness across continuous, a 100ms window, and a 250ms window, replaying 2M pings from a 40k-device fleet at ~80k events/sec against a single evaluation node. Staleness is measured event-time to commit-time; CPU is time.process_time per event.

import time

def bench_window(stream, eval_fn) -> dict[str, float]:
    t0 = time.process_time()
    stale: list[float] = []
    for committed in eval_fn(stream):
        stale.append((committed.commit_ns - committed.event_time_ns) / 1e6)  # ms
    cpu_us = (time.process_time() - t0) / len(stream) * 1e6
    stale.sort()
    return {
        "cpu_us_per_event": cpu_us,
        "p95_stale_ms": stale[int(0.95 * len(stale))],
        "p99_stale_ms": stale[int(0.99 * len(stale))],
    }

The 250ms window cuts CPU-per-event from 14.2µs to 2.5µs — a 5.6× reduction — and lifts the throughput ceiling from 70k to 260k events/sec, while holding P99 staleness at 258ms, safely inside a 500ms deadline. The 100ms window is the middle ground: 3.6× CPU savings at a 121ms P99 staleness, the right pick when the deadline is tighter. Verify before promoting by replaying against the ground-truth transition log and confirming the window introduces zero missed transitions (it must not — batching changes when a trigger fires, never whether it fires), then watch P99 staleness against the deadline as you widen the window in staging.

Failure modes and edge cases #

• Window too large → stale triggers. If $\tau$ pushes $\tau /2+t_{\text{proc}}$ past the deadline $D$, triggers fire after the physical state they describe has already advanced — a vehicle bills for a surge zone it already left. Clamp $\tau \le 2(D-t_{\text{proc}})$ using the P99 of proc_ms, and alert when measured P99 staleness exceeds 80% of the deadline.
• Traffic spike overruns the window. A pure time-flush lets a burst pack one window with tens of thousands of events, inflating both batch memory and staleness. The max_batch size cap must co-bound the flush so the tighter of time-or-size wins.
• Empty windows on idle streams. A silent device produces empty flushes that still pay the fixed overhead $o$. Skip the batch path when buf is empty, and let an idle timer handle final DWELL commits rather than spinning empty windows.
• GC pressure from batch buffers. Each flush allocates a fresh coordinate array; at high flush rates these transient arrays churn generation 0. Reuse a pre-allocated NumPy buffer sized to max_batch and apply the memory-constrained processing discipline — gc.freeze() after warm-up, buffer pooling — so a flush allocates nothing steady-state.
• Late events crossing a window boundary. A ping that arrives after its window flushed is late and must go to the watermark path, not a later window — folding it into the next batch would resolve its transition out of order. Route it through the late-telemetry and watermark side-output.

Related #

• Streaming vs Batch Geofence Evaluation — the parent execution-model decision this window-size trade sits inside.
• Late-Arriving Telemetry and Watermarks in Geofence Streams — where events that cross a window boundary are reconciled.
• Event-Time Ordering & Clock Skew in Geofence Triggers — the ordering guarantee each window’s batch depends on.
• Core Architecture & Latency Constraints — where window staleness fits in the end-to-end latency budget.