Choosing Hysteresis Buffer Widths from GPS Error Distributions
The hysteresis band described in boundary hysteresis and debounce tuning is only as good as the number chosen for its half-width, and that number is a quantile of a distribution most teams never actually look at. The standard shortcut — take the reported accuracy, call it a standard deviation, multiply by three — assumes GPS error is Gaussian. It is not. Its tail is heavier by a factor that matters: on the fleet measured for this page the empirical 99.9th percentile of horizontal error was 3.4× what a normal fit predicted, and a band sized on the normal assumption leaked eight times the false transitions it was designed to allow. This page sits under the hysteresis cluster and within the wider Core Architecture & Latency Constraints section, and it replaces the shortcut with a measurement.
Concept and specification
The band half-width $h$ must be large enough that a device genuinely sitting at the nominal boundary produces a wrong-side fix on no more than a fraction $p$ of its reports. Formally, if $E$ is the random variable for horizontal position error projected onto the axis perpendicular to the boundary, then
where is the quantile function of $E$. Nothing in that statement requires a distributional family, and the whole argument of this page is that assuming one is where the error enters.
Three properties of real GPS error break the Gaussian assumption. Multipath in urban environments produces a small population of fixes displaced by tens of metres with no corresponding change in reported accuracy, which is a mixture rather than a tail. Satellite-geometry changes — a constellation shift as a vehicle turns into a street canyon — cause step changes in error that persist for tens of seconds, so errors are correlated in time rather than independent. And receivers report accuracy as a 68% confidence radius under their own model, which is a claim about the distribution rather than a measurement of it.
| Parameter | Symbol | Typical value | Effect if mis-sized |
|---|---|---|---|
| Target false-transition rate | $p$ | 0.001 | Too high: flapping persists; too low: band swallows small fences |
| Empirical band half-width | $h$ | 25–40 m | Set below the empirical quantile, leaks duplicates |
| Reported accuracy quantile | — | P68 by convention | Treating it as a standard deviation understates the tail |
| Perpendicular projection factor | — | 0.71–1.0 | Ignoring it over-sizes the band by up to 41% |
| Sample window for the fit | — | 7 days | Under 24 h misses the diurnal constellation cycle |
The projection factor is the one correction that makes the band smaller rather than larger, and it is worth taking. Positional error is two-dimensional, but only the component perpendicular to the boundary can move a fix across it. For a boundary of arbitrary orientation and isotropic error, the perpendicular component of a radial error $r$ has standard deviation , so sizing the band from the radial quantile over-provisions by about 41%. Where the boundary orientation is known — most fences are polygons whose local edge direction is available — the projection can be applied per edge.
Step-by-step implementation
Prerequisites: Python 3.11+, numpy>=1.26, a week of raw fixes with reported accuracy, and a set of reference positions where the true location is known. The last item is the one that takes work, and there are two practical sources: devices that are provably stationary for long periods (their true position is the long-run median of their own fixes) and devices with a surveyed reference (a depot charging bay, a fixed installation).
1. Build the empirical error sample from stationary devices. A device whose fixes have a standard deviation under a threshold across a ten-minute window is stationary; its per-fix error is its displacement from the window’s geometric median, which is robust to the multipath outliers that would drag a mean.
from __future__ import annotations
import numpy as np
def stationary_errors(xy: np.ndarray, win: int = 600, still_m: float = 4.0) -> np.ndarray:
"""Per-fix displacement from the window median, for windows that look still.
xy is an (n, 2) array of local metric coordinates, one row per fix."""
out: list[np.ndarray] = []
for i in range(0, len(xy) - win, win):
block = xy[i:i + win]
centre = np.median(block, axis=0) # robust to multipath spikes
d = np.linalg.norm(block - centre, axis=1)
if np.percentile(d, 50) < still_m: # the window is genuinely still
out.append(d)
return np.concatenate(out) if out else np.empty(0)
2. Compare the empirical quantile against the Gaussian one. This is the diagnostic that justifies the whole exercise; run it before changing any configuration.
def tail_ratio(err: np.ndarray, p: float = 0.001) -> tuple[float, float, float]:
"""(empirical quantile, Gaussian-fit quantile, ratio) at the 1-p level."""
from scipy.stats import norm
emp = float(np.quantile(err, 1.0 - p))
# A radial error's Gaussian equivalent: fit sigma from the robust spread.
sigma = float(np.median(err) / 1.1774) # median of a Rayleigh ~ 1.1774 sigma
gauss = float(norm.ppf(1.0 - p) * sigma)
return emp, gauss, emp / gauss
3. Bucket by accuracy tier before taking the quantile. A single quantile over a mixed sample describes no device. Bucket the sample by the accuracy the receiver reported and take a quantile per bucket, so the runtime can select a band from the accuracy attached to each fix.
4. Apply the perpendicular projection and clamp to the fence. Divide the radial quantile by for an unknown boundary orientation, then clamp the result so the band cannot consume the fence: a practical ceiling is a quarter of the fence’s inradius, below which buffer(-h) starts producing degenerate geometry.
import math
def band_half_width(radial_q: float, fence_inradius_m: float) -> float:
perpendicular = radial_q / math.sqrt(2.0)
return min(perpendicular, 0.25 * fence_inradius_m)
5. Re-fit weekly and alert on drift. The distribution moves with the fleet: new device models, a firmware change to the location provider, a shift from urban to motorway duty. Re-run the fit weekly and alert when the P99.9 for any accuracy tier moves by more than 20%, which is nearly always a hardware or SDK change rather than an environmental one.
Benchmark and verification
Measured over seven days, 4,000 devices, 61 million fixes, against surveyed reference positions where available and stationary-window medians elsewhere:
| Sizing method | Band half-width | False transitions | Missed real transitions | Added latency P50 |
|---|---|---|---|---|
| Reported accuracy × 1 | 8 m | 4.10% | 0.00% | 0.9 s |
| Gaussian fit, 3σ | 21 m | 0.74% | 0.01% | 2.5 s |
| Empirical P99.9, radial | 48 m | 0.04% | 0.31% | 5.8 s |
| Empirical P99.9, perpendicular | 34 m | 0.09% | 0.09% | 4.1 s |
| Per-tier empirical, perpendicular | 12–61 m | 0.08% | 0.03% | 2.2 s |
The last two rows are the result worth keeping. Applying the perpendicular projection to the empirical quantile cuts the band from 48 m to 34 m and cuts missed real transitions by 3.4× while costing only 0.05 percentage points of false ones — because the 48 m band was over-provisioned in the first place. And bucketing by accuracy tier beats every fixed band on both error axes simultaneously, at half the latency of the fixed empirical band, because most fixes are good and only the poor ones need a wide band.
The verification that matters is a replay: run a captured week through the candidate band and the current one, and diff the transition streams. Every transition present in one and absent in the other should be attributable to a specific fix, and those fixes should be inspectable. A change that alters transitions it cannot explain is a bug, not a tuning improvement.
Failure modes and edge cases
| Failure mode | Signature | Mitigation |
|---|---|---|
| Fitting on moving devices | Band 3–5× too wide; error sample dominated by real motion | Restrict the sample to stationary windows by median displacement |
| Trusting reported accuracy | Band correct for 68% of fixes, far too small for the rest | Bucket by reported accuracy, take the quantile within each bucket |
| Sample shorter than 24 h | Band drifts weekly; misses the diurnal constellation cycle | Fit over at least seven days |
| Band exceeds the fence inradius | buffer(-h) returns empty or multipart geometry |
Clamp to a quarter of the inradius; flag the fence accuracy-limited |
| Indoor or Wi-Fi-derived fixes in the sample | Tail inflated by two orders of magnitude | Exclude fixes whose provider is not GNSS, or bucket them separately |
| Assuming errors are independent | Confirmation windows sized too short | Measure the autocorrelation; size the window from run length, not from per-fix probability |
Two of these deserve a sentence more. Correlated error is the reason a confirmation window cannot be derived from the same fit: if wrong-side fixes were independent, a 3-of-5 window would suppress of them, but measured runs of consecutive wrong-side fixes have a P95 length of four, so the window has to be sized from run length rather than from the marginal probability. And the indoor case is not a tail at all but a different distribution — a Wi-Fi fix in a warehouse can be 400 m out with a reported accuracy of 30 m — which is why the accuracy tiers in Wi-Fi and cell-tower fallback accuracy tiers must be kept separate rather than pooled.
Finally, a degenerate case worth guarding: a fence smaller than the band has no interior after the inward offset, and the correct behaviour is to fall back to confirmation-only damping rather than to a band of zero. A band of zero is indistinguishable at runtime from no hysteresis, and the resulting flapping will be attributed to the debouncer rather than to the fence being too small for the fleet’s accuracy.
Related
- Boundary Hysteresis & Debounce Tuning — the parent topic, where the band and the confirmation window are combined.
- Debouncing Boundary Flapping with State Machines — sizing the confirmation window from run length rather than from this fit.
- Wi-Fi and Cell-Tower Fallback Accuracy Tiers — the tiers this fit must be bucketed by.