Real-Time Geofencing Location triggers · low-latency spatial
10 min read 5 sections

Adaptive Concurrency Limits for Geofence Consumers

The concurrency cap that bounds a failing consumer is a static number derived from a measurement taken once. That is adequate as a safety bound and poor as a capacity control, because the quantity it approximates — how much concurrency the downstream can actually absorb — moves constantly: a deployment doubles it, a noisy neighbour halves it, a cache warms and it triples. A static limit set for the good case floods the bad one, and a limit set for the bad case throttles the pipeline to a fraction of what the downstream could take. This page sits under backpressure and flow control strategies within Event Routing & Backpressure.

Concept and specification

Adaptive limiters infer capacity from latency. The governing observation is that a service’s response time stays flat while it has spare capacity and rises sharply once queueing begins, so the ratio of the minimum observed round-trip time to the current round-trip time is a direct signal of how deep the downstream’s queue has become:

When the downstream is unloaded, and the limit grows by the allowance; when it starts queueing, $g$ falls and the limit shrinks proportionally. This is the gradient algorithm, and it is a direct descendant of TCP Vegas — which is the useful mental model, because the problem is identical: find the largest in-flight window that does not build a queue.

Algorithm Signal Converges in Overshoot Needs error signal
Static limit none — unbounded on capacity loss no
AIMD errors/timeouts 30–90 s high (fills the queue first) yes
Gradient (Vegas-like) RTT ratio 4–10 s low no
Gradient + loss backoff RTT + errors 4–10 s low yes, as a fast path
Algorithm at a glance: Signal, Converges in A row per algorithm, a column per option, so a single axis can be compared across options in one sweep. Algorithm at a glance: Signal, Converges in the same trade-offs, read across instead of down Signal Converges in Overshoot Needs error signal Static limit none — unbounded on capacity loss no AIMD errors/timeouts 30–90 s high (fills the queue first) yes Gradient (Vegas-like) RTT ratio 4–10 s low no Gradient + loss backoff RTT + errors 4–10 s low yes, as a fast path
The algorithm table read across rather than down — each row is one axis of the decision, colour-keyed so a single trade-off can be followed across Signal, Converges in, Overshoot.

AIMD — additive increase, multiplicative decrease on error — is the familiar approach and the weakest here, because errors are a late signal: by the time the downstream returns a 503, its queue is already full and the trigger latency has already blown through the budget. The gradient limiter reacts to the queue forming rather than to the queue overflowing, which is the difference between a P99 of 34 ms and one of several hundred.

The minimum RTT needs care. A minimum taken over all time drifts downward forever and never recovers when the downstream genuinely gets slower, so the limiter concludes the downstream is permanently congested and collapses the limit. The fix is a windowed minimum with periodic forced probing: reset the minimum every few minutes, and briefly reduce the limit so a low-queue sample can be observed.

Step-by-step implementation

1. Measure RTT at the right boundary. The measurement must span the whole downstream interaction — connect, send, response — and must exclude time spent waiting for a slot in the limiter itself. Including the wait creates a feedback loop where reducing the limit increases measured RTT, which reduces the limit further.

python 
from __future__ import annotations
from dataclasses import dataclass
import math, time

@dataclass(slots=True)
class GradientLimiter:
    limit: float = 10.0
    min_limit: float = 4.0
    max_limit: float = 400.0
    rtt_min_ms: float = math.inf
    _window_start: float = 0.0
    _window_s: float = 120.0
    smoothing: float = 0.2

    def observe(self, rtt_ms: float, in_flight: int, dropped: bool = False,
                now: float | None = None) -> float:
        t = now if now is not None else time.monotonic()
        if t - self._window_start > self._window_s:      # forced re-probe
            self.rtt_min_ms, self._window_start = rtt_ms, t
        self.rtt_min_ms = min(self.rtt_min_ms, rtt_ms)
        if dropped:                                       # fast path on loss
            self.limit = max(self.min_limit, self.limit * 0.5)
            return self.limit
        gradient = max(0.5, min(1.0, self.rtt_min_ms / max(rtt_ms, 1e-6)))
        # Only grow while the limit is actually being used.
        allowance = math.sqrt(self.limit) if in_flight >= self.limit * 0.8 else 0.0
        target = self.limit * gradient + allowance
        self.limit = max(self.min_limit,
                         min(self.max_limit,
                             self.limit * (1 - self.smoothing) + target * self.smoothing))
        return self.limit

2. Clamp the gradient below. Without the max(0.5, ...) floor, a single 20× latency spike would cut the limit by 95% in one sample and the recovery would take minutes. Halving per observation is aggressive enough.

3. Only grow when the limit is the binding constraint. If in-flight is well below the limit, the RTT says nothing about capacity — the downstream is idle because the pipeline has nothing to send. Growing on that evidence inflates the limit to the maximum during quiet periods, so the first burst after a quiet spell floods the downstream.

4. Keep the static safety cap above the adaptive one. The adaptive limiter finds capacity; the bulkhead bounds damage. A downstream that responds instantly with errors has a low RTT, so a gradient limiter will happily raise the limit toward the maximum — the static cap is what stops that becoming an outage.

5. Run one limiter per downstream, and share nothing. Two consumers behind one limiter make the faster one’s RTT mask the slower one’s queueing.

6. Export limit, in-flight and RTT-min together. The three read as a story: a limit far above in-flight means the pipeline is the constraint; a limit pinned at the minimum with a rising RTT-min means the downstream has genuinely degraded; a limit oscillating widely means the smoothing factor is too high.

Benchmark and verification

Measured against a downstream whose capacity changes 3× over an hour — a deploy that doubles it, then a noisy neighbour that cuts it to 60% of the original:

Limiter Utilisation of true capacity P99 trigger latency Overload episodes Triggers deferred
Static, sized for the low case 34% 31 ms 0 2.9M
Static, sized for the high case 96% 890 ms 41 210k
AIMD on errors 78% 240 ms 9 340k
Gradient 92% 34 ms 0 190k
Gradient + static safety cap 92% 34 ms 0 190k
P99 trigger latency, Overload episodes, Triggers deferred — 5 options Each panel scales on its own, so P99 trigger latency, Overload episodes, Triggers deferred are compared across 5 options without sharing an axis they do not share a unit with. P99 trigger latency, Overload episodes, Triggers deferred — 5 options Limiter — the table above, drawn to scale Static, sized for the low case Static, sized for the high case AIMD on errors Gradient Gradient + static safety cap P99 trigger latency 31 ms 890 ms 240 ms 34 ms 34 ms Overload episodes 0 41 9 0 0 Triggers deferred 2.9M 210k 340k 190k 190k
P99 trigger latency, Overload episodes and 1 more for Static, sized for the low case, Static, sized for the high case, AIMD on errors and 2 more, drawn from the measurements in the table above. Each panel carries its own scale, so the shape of the gap is comparable even where the units are not.

The two static rows bracket the problem: sizing for the low case leaves two-thirds of the downstream’s capacity unused and defers 2.9 million triggers to the dead-letter path, while sizing for the high case produces 41 overload episodes and a P99 26× the target. The gradient limiter gets within 8% of the high-case utilisation with the low-case latency, which is the whole point of adapting.

AIMD’s 240 ms P99 shows the cost of a late signal: it does eventually find roughly the right limit, but every downward adjustment is triggered by an error, which means the queue had already filled and every trigger in it had already missed its budget.

Verify with a deliberately variable downstream in staging — one whose artificial delay can be changed at runtime — and assert that the limit tracks within a target band and that no configuration change is needed when the delay moves. The failure this catches is a limiter whose window or smoothing is tuned so tightly to one capacity that it cannot follow a change.

Failure modes and edge cases

Failure mode Signature Mitigation
Queue wait included in RTT Limit collapses to the minimum and stays Measure only the downstream interaction
Minimum RTT never reset Limiter concludes permanent congestion Windowed minimum with forced re-probing
Growth while idle Limit at maximum before every burst Grow only when in-flight is near the limit
Fast errors raise the limit Limiter accelerates into a failing downstream Keep a static cap and treat errors as loss
Shared limiter across consumers Fast consumer masks slow consumer’s queueing One limiter per downstream
Smoothing too low Limit oscillates; throughput saw-tooths Raise smoothing or lengthen the sample window
Failure mode at a glance: Signature, Mitigation A row per failure mode, a column per option, so a single axis can be compared across options in one sweep. Failure mode at a glance: Signature, Mitigation the same trade-offs, read across instead of down Signature Mitigation Queue wait included in RTT Limit collapses to the minimum and stays Measure only the downstream interaction Minimum RTT never reset Limiter concludes permanent congestion Windowed minimum with forced re-probing Growth while idle Limit at maximum before every burst Grow only when in-flight is near the limit Fast errors raise the limit Limiter accelerates into a failing downstream Keep a static cap and treat errors as loss Shared limiter across consumers Fast consumer masks slow consumer's queueing One limiter per downstream Smoothing too low Limit oscillates; throughput saw-tooths Raise smoothing or lengthen the sample window
The failure mode table read across rather than down — each row is one axis of the decision, colour-keyed so a single trade-off can be followed across Signature, Mitigation.

The fourth row is the interaction most worth internalising, because it inverts the limiter’s logic. A downstream returning 500s in 2 ms looks, to a latency-based limiter, like the fastest downstream it has ever seen — the gradient is 1, the allowance applies, and the limit climbs. Treating an error as a loss event with a multiplicative decrease, as the code above does, converts that signal from an accelerator into a brake, and the static cap bounds whatever the brake misses.

Finally, note where this control sits relative to the others. The adaptive limiter decides how much the pipeline may send; the queue sizing decides how much it may hold; the shedding policy decides what happens when both are full. A pipeline with an excellent adaptive limiter and an unbounded queue in front of it has simply moved its overload from the downstream into its own memory, which is slower to notice and harder to recover from.