Validation Scoreboard

We score our own output. Every day we compare this system's digest against expert human analysis — ISW's Russian Offensive Campaign Assessment and other baselines — measuring whether we surfaced the same events, how early, and how much of what we published rests on more than one source. We publish the results, including the misses, because analysts should know exactly how much to trust an automated feed. Unlike a finished prose report, every claim here links back to its source document, is searchable, and can land hours earlier — this page shows what that speed costs in coverage.

• Coverage % — the share of ISW's same-day takeaways our digest also matched. The headline number scores our finalized digest (published ~10:00 PM ET), after ISW's report is already out; a row marked “intraday cut” scores the last intraday version instead, and one marked “regenerated after scoring” was scored before its digest was rewritten.
• Evidence available at ISW publish (proxy) — of the same matched takeaways, the share whose supporting evidence we had already ingested before ISW published. It gauges what evidence was in hand at that moment; it does not prove the claim appeared in our published digest then. RU and UA are currently scored as separate country digests against the same ROCA report and denominator.
• Information lead — median hours between our earliest supporting source document and ISW's publish time, across matched events; positive means we had it first. Pages read from criticalthreats.org carry no publish time: Russia assessments are timed from ISW's own “assessment as of” line, a lower bound on its publish time, so their lead and the median read “≥”; Iran Update pages have no such line, so those runs carry no lead and the median leaves them out.
• Thin-sourced % — the share of our claims resting on a single source while still hedged as claimed or unverified, never stated as settled fact. Lower is better.
• Agreement / ISW-only / ours-only — events both sides reported, events ISW reported that we missed, and events we reported that ISW didn't carry. An ours-only item counts as a potential lead only when confirmed; unconfirmed items are labeled with their hedge.