11 min read 5 sections

Choosing Hysteresis Buffer Widths from GPS Error Distributions

The hysteresis band described in boundary hysteresis and debounce tuning is only as good as the number chosen for its half-width, and that number is a quantile of a distribution most teams never actually look at. The standard shortcut — take the reported accuracy, call it a standard deviation, multiply by three — assumes GPS error is Gaussian. It is not. Its tail is heavier by a factor that matters: on the fleet measured for this page the empirical 99.9th percentile of horizontal error was 3.4× what a normal fit predicted, and a band sized on the normal assumption leaked eight times the false transitions it was designed to allow. This page sits under the hysteresis cluster and within the wider Core Architecture & Latency Constraints section, and it replaces the shortcut with a measurement.

Concept and specification

The band half-width $h$ must be large enough that a device genuinely sitting at the nominal boundary produces a wrong-side fix on no more than a fraction $p$ of its reports. Formally, if $E$ is the random variable for horizontal position error projected onto the axis perpendicular to the boundary, then

where is the quantile function of $E$. Nothing in that statement requires a distributional family, and the whole argument of this page is that assuming one is where the error enters.

Three properties of real GPS error break the Gaussian assumption. Multipath in urban environments produces a small population of fixes displaced by tens of metres with no corresponding change in reported accuracy, which is a mixture rather than a tail. Satellite-geometry changes — a constellation shift as a vehicle turns into a street canyon — cause step changes in error that persist for tens of seconds, so errors are correlated in time rather than independent. And receivers report accuracy as a 68% confidence radius under their own model, which is a claim about the distribution rather than a measurement of it.

Parameter Symbol Typical value Effect if mis-sized
Target false-transition rate $p$ 0.001 Too high: flapping persists; too low: band swallows small fences
Empirical band half-width $h$ 25–40 m Set below the empirical quantile, leaks duplicates
Reported accuracy quantile P68 by convention Treating it as a standard deviation understates the tail
Perpendicular projection factor 0.71–1.0 Ignoring it over-sizes the band by up to 41%
Sample window for the fit 7 days Under 24 h misses the diurnal constellation cycle
Parameter at a glance: Symbol, Typical value A row per parameter, a column per option, so a single axis can be compared across options in one sweep. Parameter at a glance: Symbol, Typical value the same trade-offs, read across instead of down Symbol Typical value Effect if mis-sized Target false-transition rate p 0.001 Too high: flapping persists; too low: band swallows small fences Empirical band half-width h 25–40 m Set below the empirical quantile, leaks duplicates Reported accuracy quantile P68 by convention Treating it as a standard deviation understates the tail Perpendicular projection factor 0.71–1.0 Ignoring it over-sizes the band by up to 41% Sample window for the fit 7 days Under 24 h misses the diurnal constellation cycle
The parameter table read across rather than down — each row is one axis of the decision, colour-keyed so a single trade-off can be followed across Symbol, Typical value, Effect if mis-sized.

The projection factor is the one correction that makes the band smaller rather than larger, and it is worth taking. Positional error is two-dimensional, but only the component perpendicular to the boundary can move a fix across it. For a boundary of arbitrary orientation and isotropic error, the perpendicular component of a radial error $r$ has standard deviation , so sizing the band from the radial quantile over-provisions by about 41%. Where the boundary orientation is known — most fences are polygons whose local edge direction is available — the projection can be applied per edge.

Step-by-step implementation

Prerequisites: Python 3.11+, numpy>=1.26, a week of raw fixes with reported accuracy, and a set of reference positions where the true location is known. The last item is the one that takes work, and there are two practical sources: devices that are provably stationary for long periods (their true position is the long-run median of their own fixes) and devices with a surveyed reference (a depot charging bay, a fixed installation).

1. Build the empirical error sample from stationary devices. A device whose fixes have a standard deviation under a threshold across a ten-minute window is stationary; its per-fix error is its displacement from the window’s geometric median, which is robust to the multipath outliers that would drag a mean.

python
from __future__ import annotations
import numpy as np

def stationary_errors(xy: np.ndarray, win: int = 600, still_m: float = 4.0) -> np.ndarray:
    """Per-fix displacement from the window median, for windows that look still.
    xy is an (n, 2) array of local metric coordinates, one row per fix."""
    out: list[np.ndarray] = []
    for i in range(0, len(xy) - win, win):
        block = xy[i:i + win]
        centre = np.median(block, axis=0)          # robust to multipath spikes
        d = np.linalg.norm(block - centre, axis=1)
        if np.percentile(d, 50) < still_m:         # the window is genuinely still
            out.append(d)
    return np.concatenate(out) if out else np.empty(0)

2. Compare the empirical quantile against the Gaussian one. This is the diagnostic that justifies the whole exercise; run it before changing any configuration.

python
def tail_ratio(err: np.ndarray, p: float = 0.001) -> tuple[float, float, float]:
    """(empirical quantile, Gaussian-fit quantile, ratio) at the 1-p level."""
    from scipy.stats import norm
    emp = float(np.quantile(err, 1.0 - p))
    # A radial error's Gaussian equivalent: fit sigma from the robust spread.
    sigma = float(np.median(err) / 1.1774)         # median of a Rayleigh ~ 1.1774 sigma
    gauss = float(norm.ppf(1.0 - p) * sigma)
    return emp, gauss, emp / gauss

3. Bucket by accuracy tier before taking the quantile. A single quantile over a mixed sample describes no device. Bucket the sample by the accuracy the receiver reported and take a quantile per bucket, so the runtime can select a band from the accuracy attached to each fix.

4. Apply the perpendicular projection and clamp to the fence. Divide the radial quantile by for an unknown boundary orientation, then clamp the result so the band cannot consume the fence: a practical ceiling is a quarter of the fence’s inradius, below which buffer(-h) starts producing degenerate geometry.

python
import math

def band_half_width(radial_q: float, fence_inradius_m: float) -> float:
    perpendicular = radial_q / math.sqrt(2.0)
    return min(perpendicular, 0.25 * fence_inradius_m)

5. Re-fit weekly and alert on drift. The distribution moves with the fleet: new device models, a firmware change to the location provider, a shift from urban to motorway duty. Re-run the fit weekly and alert when the P99.9 for any accuracy tier moves by more than 20%, which is nearly always a hardware or SDK change rather than an environmental one.

Benchmark and verification

Measured over seven days, 4,000 devices, 61 million fixes, against surveyed reference positions where available and stationary-window medians elsewhere:

Sizing method Band half-width False transitions Missed real transitions Added latency P50
Reported accuracy × 1 8 m 4.10% 0.00% 0.9 s
Gaussian fit, 3σ 21 m 0.74% 0.01% 2.5 s
Empirical P99.9, radial 48 m 0.04% 0.31% 5.8 s
Empirical P99.9, perpendicular 34 m 0.09% 0.09% 4.1 s
Per-tier empirical, perpendicular 12–61 m 0.08% 0.03% 2.2 s
False transitions, Missed real transitions, Added latency P50 — 5 options Each panel scales on its own, so False transitions, Missed real transitions, Added latency P50 are compared across 5 options without sharing an axis they do not share a unit with. False transitions, Missed real transitions, Added latency P50 — 5 options Sizing method — the table above, drawn to scale Reported accuracy × 1 Gaussian fit, 3σ Empirical P99.9, radial Empirical P99.9, perpendicular Per-tier empirical, perpendicular False transitions 4.10% 0.74% 0.04% 0.09% 0.08% Missed real transitions 0.00% 0.01% 0.31% 0.09% 0.03% Added latency P50 0.9 s 2.5 s 5.8 s 4.1 s 2.2 s
False transitions, Missed real transitions and 1 more for Reported accuracy × 1, Gaussian fit, 3σ, Empirical P99.9, radial and 2 more, drawn from the measurements in the table above. Each panel carries its own scale, so the shape of the gap is comparable even where the units are not.

The last two rows are the result worth keeping. Applying the perpendicular projection to the empirical quantile cuts the band from 48 m to 34 m and cuts missed real transitions by 3.4× while costing only 0.05 percentage points of false ones — because the 48 m band was over-provisioned in the first place. And bucketing by accuracy tier beats every fixed band on both error axes simultaneously, at half the latency of the fixed empirical band, because most fixes are good and only the poor ones need a wide band.

The verification that matters is a replay: run a captured week through the candidate band and the current one, and diff the transition streams. Every transition present in one and absent in the other should be attributable to a specific fix, and those fixes should be inspectable. A change that alters transitions it cannot explain is a bug, not a tuning improvement.

Failure modes and edge cases

Failure mode Signature Mitigation
Fitting on moving devices Band 3–5× too wide; error sample dominated by real motion Restrict the sample to stationary windows by median displacement
Trusting reported accuracy Band correct for 68% of fixes, far too small for the rest Bucket by reported accuracy, take the quantile within each bucket
Sample shorter than 24 h Band drifts weekly; misses the diurnal constellation cycle Fit over at least seven days
Band exceeds the fence inradius buffer(-h) returns empty or multipart geometry Clamp to a quarter of the inradius; flag the fence accuracy-limited
Indoor or Wi-Fi-derived fixes in the sample Tail inflated by two orders of magnitude Exclude fixes whose provider is not GNSS, or bucket them separately
Assuming errors are independent Confirmation windows sized too short Measure the autocorrelation; size the window from run length, not from per-fix probability
Failure mode at a glance: Signature, Mitigation A row per failure mode, a column per option, so a single axis can be compared across options in one sweep. Failure mode at a glance: Signature, Mitigation the same trade-offs, read across instead of down Signature Mitigation Fitting on moving devices Band 3–5× too wide; error sample dominated by real motion Restrict the sample to stationary windows by median displacement Trusting reported accuracy Band correct for 68% of fixes, far too small for the rest Bucket by reported accuracy, take the quantile within each bucket Sample shorter than 24 h Band drifts weekly; misses the diurnal constellation cycle Fit over at least seven days Band exceeds the fence inradius buffer(-h) returns empty or multipart geometry Clamp to a quarter of the inradius; flag the fence accuracy-limited Indoor or Wi-Fi-derived fixes in the sample Tail inflated by two orders of magnitude Exclude fixes whose provider is not GNSS, or bucket them separately Assuming errors are independent Confirmation windows sized too short Measure the autocorrelation; size the window from run length, not from per-fix probability
The failure mode table read across rather than down — each row is one axis of the decision, colour-keyed so a single trade-off can be followed across Signature, Mitigation.

Two of these deserve a sentence more. Correlated error is the reason a confirmation window cannot be derived from the same fit: if wrong-side fixes were independent, a 3-of-5 window would suppress of them, but measured runs of consecutive wrong-side fixes have a P95 length of four, so the window has to be sized from run length rather than from the marginal probability. And the indoor case is not a tail at all but a different distribution — a Wi-Fi fix in a warehouse can be 400 m out with a reported accuracy of 30 m — which is why the accuracy tiers in Wi-Fi and cell-tower fallback accuracy tiers must be kept separate rather than pooled.

Finally, a degenerate case worth guarding: a fence smaller than the band has no interior after the inward offset, and the correct behaviour is to fall back to confirmation-only damping rather than to a band of zero. A band of zero is indistinguishable at runtime from no hysteresis, and the resulting flapping will be attributed to the debouncer rather than to the fence being too small for the fleet’s accuracy.