Internet measurement often requires inferring what we cannot directly observe: locations, dependencies, physical routes, and failures. Under partial observability, different realities may fit the same evidence, and ground truth may be scarce, selective, or inaccessible. What can we responsibly claim?
Partial observability is the problem
Partial observability is more than a shortage of labels. Measurements may leave multiple explanations consistent with the evidence; vantage points may systematically miss important behavior; and additional observations of the same kind may not resolve the ambiguity. Reporting only an inferred answer conceals what the evidence supports—and what remains unknown.
Self-validating uncertainty is our proposed response: a design principle for inference-based measurement under partial observability. Systems should expose not only inferences, but also the evidence, uncertainty, and assumptions needed to assess and challenge their reliability claims. This does not mean a system can certify its own answers or make every limitation observable.
GeolocationWhere is a router or network path, and what evidence supports the inferred location?
Infrastructure mappingWhich physical cables or shared dependencies might underlie an observed logical connection?
Failure inferenceWhat can observations establish about a disruption when internal operator state is unavailable?
One concrete case study: We use Internet-path geolocation to investigate how scarce ground truth can support empirical reliability estimates under partial observability. Other settings—including submarine-cable mapping and failure inference—may require different forms of uncertainty, such as sets of plausible explanations or falsifiable consistency checks. The experiment demonstrates both the potential and the limitations of one method; self-validating uncertainty is the broader design principle.
Case study · Inferred Internet-path geolocation
≈408,000
Inferred Internet paths in the corpus
4,678
Paths in the available ground-truth pool (≈1.1% of corpus)
A concrete example: from individual answers to testable reliability claims
If we validate a path, we learn whether that path is correct. But most paths cannot be checked. Our approach asks a different question: for inferences sharing similar observable evidence, how often are they correct?
1 · Expose a signalFor every inferred path, compute ℛ, a path–model alignment score requiring no ground truth. ℛ is not a correctness probability.
2 · Learn from scarce labelsGroup paths by ℛ and use validated examples to estimate each group's observed correctness frequency, with uncertainty intervals.
3 · Challenge the claimAsk how many paths can be characterized—and whether the labeled examples resemble the paths to which we hope to apply the estimate.
A group's reliability estimate tells us how often its members were correct in the labeled sample. It does not identify which individual paths are wrong.
This experiment studies one consequence of partial observability: only some inferred Internet-path locations can be checked against reference locations. Its score and validation procedure are specific to this case study, not requirements for other measurement systems.
What is the corpus?
Approximately 408,000 paths with inferred geographic locations. The experiment asks what can be said about these inferences without individually checking every path.
What is the labeled pool?
4,678 paths with reference geolocation information obtained through the study's active validation process, using available measurement vantage points. Availability is selective: shorter paths are easier to validate. This pool is not a random sample of the corpus.
What is ℛ?
Path–Model Alignment, a score in [0, 1] measuring agreement between latency–distance residual profiles of the inferred path and the original geolocation-database path. It requires no ground truth and is not a probability of correctness. Paths are grouped into ten fixed-width ℛ intervals.
What counts as correct?
A labeled path has y = 1 when its mean path geolocation error is ≤ 200 km; otherwise y = 0. Within each ℛ group, the observed fraction of correct labels estimates group-level correctness, not which particular unlabeled paths are correct.
Interpretation depends on assumptions: labels must be informative for the group being characterized; intervals must account for the sampling and dependence structure; and transferring estimates to the corpus requires evidence that correctness relationships persist beyond the labeled pool. These assumptions are examined in the three views, not silently guaranteed.
01 · HOW FAR CAN LIMITED GROUND TRUTH TAKE US?
The payoff: how far can a small label budget reach?
Individual validation tells us about the paths we directly check. Rather than establishing correctness one inference at a time, group-level reliability estimation characterizes correctness rates across much larger portions of the corpus—without identifying which individual inferences are correct.
Try this Reduce the label budget and compare how much of the corpus each approach covers. Switch the sampling regime to see how the result changes.
Individually validated share of corpus
Median share in groups meeting the precision criterion
Coverage gap (percentage points)
Group reliability coverageIndividual validation5–95% simulation range
What to notice
What this does not establish Whether correctness rates estimated from the labeled pool transfer to the full corpus. Coverage means a group's correctness frequency can be estimated from at least 10 sampled labels with a Wilson 95% interval half-width ≤ 0.10. It does not mean every path in that group is correct, or that the estimate necessarily generalizes to the corpus.
02 · WHY CAN SOME GROUPS BE CHARACTERIZED AND OTHERS NOT?
Where do the labels go?
Explore a specific simulation draw at the selected budget. Groups are fixed ℛ intervals—not equal-sized corpus quantiles. The color indicates whether the group's observed estimate meets the paper's precision criterion.
Try this Select a small label budget, click different ℛ groups, and switch between biased and uniform sampling. The budget and sampling controls are shared with View 1.
Sampled labels per ℛ group
Selected group
What to notice
What this does not establish A precise group estimate need not be unbiased or representative of the corresponding corpus group. A group can receive many labels but still fail the interval-width requirement. Conversely, groups with few labels may be highly relevant to the corpus. The selected repetition is one actual draw; View 1 summarizes all 200 repetitions.
03 · CAN THE LABELED SAMPLE SPEAK FOR THE CORPUS?
What would justify transferring this estimate?
A narrow confidence interval in the labeled pool does not establish reliability in the corpus. We can examine population differences and stability across path lengths, then ask what further evidence would support transfer.
Try this Compare the pool and corpus path-length distributions, inspect the reweighted ℛ distribution, then select different ℛ groups to compare short and long paths.
A · Are the populations comparable?
Compare path-length and ℛ distributions. Reweighting is a sensitivity check, not proof that selection bias has been removed.
B · Is correctness stable across observed subpopulations?
Inspect short-versus-long correctness rates by ℛ group. Their individual confidence intervals are descriptive; overlap is not an equivalence test.
C · What evidence would justify transfer?
Specify an acceptable difference in correctness rates, obtain adequate independent labels across relevant strata, and estimate a confidence interval for the difference. An equivalence claim would require that interval to fall within the prespecified tolerance, with sampling and path dependence accounted for. Independent target-population validation would strengthen the claim.
36.7%Highest-ℛ share of labeled pool
18.8%After reweighting by hop-length class
18.4%Highest-ℛ share of corpus
Path lengths: pool vs. corpus
ℛ distribution and path-length reweighting
Correctness by path length and ℛ group
Click a group to inspect observed correctness and Wilson 95% intervals for short (≤6 hops) and long (≥9 hops) validated paths.
What to notice
What this does not establish The existing analysis does not demonstrate transfer to the full corpus. To justify an equivalence claim, specify an acceptable correctness-rate difference and show that an appropriate uncertainty interval for that difference lies inside the tolerance, accounting for sampling and dependence. Independent validation in the target population would provide stronger evidence.
Reweighting by four hop-length classes is a sensitivity analysis, not a guarantee of transfer. Exact-hop reweighting excludes corpus lengths absent from the labeled pool; roughly 6.9% of the corpus lies outside that support. The cross-length comparison is limited to validated paths, and correlated paths can make conventional intervals optimistic. No formal equivalence or transportability test is claimed here. Failure to detect a difference would not establish equivalence.
Beyond geolocation: designing for partial observability
Partial observability takes different forms across measurement domains. Not every system can group inferences by a score, and scarce labels do not always suffice to characterize an unlabeled population. Those are possibilities and limitations illustrated by this geolocation case study.
Our broader claim is that partial observability should shape what measurement systems report. Alongside inferred answers, they should expose evidence, assumptions, unresolved ambiguity, and independently checkable constraints—so others can evaluate, challenge, and sometimes falsify the claims.
Depending on the domain, this might mean empirical group-level reliability estimates, candidate physical explanations (as in submarine-cable mapping), consistency checks, or explicit unknowns. The validation dividend shows one potential benefit; the transfer analysis shows why even precise estimates cannot erase partial observability. Self-validating uncertainty does not overcome the missing evidence—it makes the boundaries of what we can claim visible.
