Back to Blog
Research

Measuring Uncertainty in Near-Term Event Forecasting

The phrase "probability estimate" carries a hidden assumption that is almost always violated in practice: that the estimate itself is a precise quantity. In reality, every probability estimate for a geopolitical or political event is derived from finite, imperfect data, subject to model uncertainty, and should be understood as a distribution rather than a point.

When we say a probability estimate for an event is 58%, we mean something closer to "our best estimate, given available evidence and the current model, is in the range of 51-65%, with 58% as the central value." The interval matters. An interval of 58% plus-or-minus 7% implies a different level of analytic confidence than 58% plus-or-minus 23%. Both intervals have the same central estimate. They do not support the same decisions.

Sources of Uncertainty in Near-Term Forecasting

Uncertainty in probability estimates for near-term events has several structurally distinct sources, and distinguishing them helps clarify what can be reduced through more data versus what cannot.

Epistemic uncertainty arises from limited information. If we have few source signals on an event, or if the signals we have come from source types with limited historical calibration for this event category, the estimate has wide uncertainty because we simply do not know enough. Epistemic uncertainty is reducible in principle: more information, or better-calibrated sources, would narrow the interval. This is the kind of uncertainty that a better OSINT aggregation layer directly addresses.

Aleatory uncertainty is irreducible. Some near-term events are genuinely difficult to predict regardless of how much information is available, because they depend on decisions by individual actors with genuine discretion, or on interactions between multiple actors whose responses to each other are not predictable from observable inputs. A ministerial decision to call an early election involves real human judgment that is not mechanically derivable from signals in the information environment. The uncertainty here is not a gap in our information collection; it is the nature of the event itself.

Model uncertainty is the third source. The Bayesian updating model that converts source signals into probability readings has its own parameters, its own assumptions about how sources interact, and its own historical calibration. Uncertainty in those parameters propagates into uncertainty in the output estimate. A source weight that is estimated from 40 resolved events carries more estimation uncertainty than one estimated from 400 resolved events, and that difference should appear in the width of the probability interval.

Expressing Uncertainty in Practical Terms

For corporate risk workflows, the most useful way to express estimate uncertainty is probably not through explicit confidence intervals in every presentation, because that level of statistical precision often exceeds what the audience will work with. The useful expressions of uncertainty are more qualitative but still structured.

One approach is source breadth reporting: alongside the central probability estimate, show how many structurally independent source categories contributed to the current reading. An estimate with contributions from four independent source types has a narrower epistemic uncertainty than an estimate with all weight concentrated in one type. The source breadth metric communicates interval width in terms the audience already understands.

Another approach is signal density tracking: how many distinct signals has the system ingested on this event over the scoring period? A high-density reading is better supported than a low-density reading at the same central probability. An event scored at 62% on 40 contributing signals means something different from the same central estimate on 8 signals.

The third approach is agreement tracking across source types. If all source categories that have contributed to the reading are pointing in the same direction, the uncertainty interval is narrower than if sources are split. Source divergence should surface as a flag on the estimate, indicating that the central probability conceals disagreement in the underlying evidence that the user should investigate.

Time Horizon and Uncertainty Width

Near-term forecasting is distinguished from longer-horizon forecasting partly by the relationship between time horizon and uncertainty. For near-term events, shorter horizons generally produce narrower uncertainty intervals because there is less time for unpredictable developments to intervene. An event scored for a 30-day resolution window has a narrower uncertainty band than the same event scored for a 180-day window, because 30 days offers fewer opportunities for exogenous shocks to change the trajectory.

This relationship has a practical implication: as a resolution date approaches, the probability estimate should generally become more stable and the uncertainty interval should narrow, assuming the event situation is not itself changing. If an estimate becomes more volatile as the resolution date approaches rather than less volatile, that suggests unstable source inputs or model sensitivity that warrants investigation.

The relationship also affects how uncertainty should be communicated to different audiences. A risk committee reviewing exposure on an event resolving in 14 days can reasonably expect tighter probability bounds than a committee reviewing an event resolving in four months. The appropriate comparison is not the absolute width of the interval but whether the interval is appropriate for the time horizon and information density available.

Uncertainty and Decision-Making Under Uncertainty

Decision theory provides frameworks for making choices under uncertainty that go beyond treating probability point estimates as certainties. The most practically relevant for corporate risk is the consideration of what decision changes as the probability interval shifts.

If a corporate exposure decision would not change regardless of whether the true probability is at the low end or the high end of the uncertainty interval, the interval width is not decision-relevant. The decision is robust to the uncertainty. If, however, different actions are optimal at different points within the interval, the uncertainty width is decision-critical, and the organization should either accept the uncertainty as a constraint on decision precision or invest in narrowing the interval through additional source coverage or expert review.

We use this framework internally when deciding how much additional source sourcing is warranted for a specific event. If the uncertainty interval is wide enough to change the decision recommendation but narrow enough that additional credible sources could meaningfully narrow it within the decision timeline, adding those sources is justified. If the uncertainty is primarily aleatory, adding sources will not help, and the organization should make the decision under explicit uncertainty rather than seeking a precision that the event type cannot support.

What We Are Still Working On

Expressing uncertainty formally in probability outputs is something we are actively developing. The current Cade Market interface shows the probability estimate and the source contribution breakdown. Explicit interval reporting, with confidence band visualization that responds to source count and calibration quality, is on the development roadmap.

The technical challenge is not the interval calculation itself, it is the calibration of the interval calculation. To produce intervals that are accurate, not just wide enough to always be correct, requires a calibrated model of how interval width relates to source count, source diversity, and historical calibration performance for specific event types. That model needs the same kind of resolved event sample that the probability calibration work requires. The two bodies of work are developing together.

Start Tracking with Calibrated Probability

Cade Market aggregates structured intelligence and expert forecasting into scored probability estimates for political and economic events.

Request Access How It Works

More Analysis

Research

Calibration Is the Missing Metric in AI Probability Scoring

Research

Confidence Intervals and Political Risk: What the Number Really Means

Research

Event-Driven Risk: Why Probability Matters to Decision-Makers