Methodology
What BNOW.NET does, requirement by requirement, against the four public Intelligence Community issuances that govern analytic tradecraft, sourcing, open-source citation, and product utility — including what we have not built.
1. How to read this page
Each requirement below is marked BUILT, PARTIAL, or GAP. BUILT means the mechanism exists in production today and a specific piece of code enforces it. PARTIAL means the inputs exist and are enforced, but the requirement is not fully met — usually because the data is held and not yet presented. GAP means it is not built. Rows we intend to close are marked planned; rows nothing on the current roadmap closes are marked not planned, rather than left ambiguous.
Of 24 requirements, 9 are BUILT, 12 are PARTIAL, and 3 are GAP. A reviewer who wants to be useful should press on the PARTIAL and GAP rows.
This page is a public summary of an internal conformance document that additionally names the file enforcing each mechanism. If you are evaluating BNOW and want that version, ask us for it.
2. Vocabulary
BNOW's product vocabulary is ICS 206-01 narrative source descriptors and ICD 203 estimative language. NATO AJP-2.1 and Admiralty-style two-axis codes — source reliability paired with information credibility — are emitted, where they are emitted at all, as derived export fields for interoperability with STIX and MISP consumers, and never as a headline rating.
The reason is that ICS 206-01 is the current standard, it was written for open-source intelligence specifically, and it is the only one of the four that addresses AI- and machine-derived inference. The Admiralty scale preserves NATO and threat-intelligence interoperability but has documented failure modes: readers collapse its two axes into a single impression of quality, and its credibility definitions are not applied consistently across users.
3. The crosswalk
Requirement text is paraphrased from the issuances. The originals are unclassified and published by the Office of the Director of National Intelligence.
ICD 203 — Analytic Standards (January 2, 2015)
| Requirement | What BNOW does | Status | Roadmap |
|---|---|---|---|
| Objective; independent of political consideration | Source standing is derived from ISW's own citation and hedging behaviour, not from an editorial preference; there is no human editorial pass between extraction and publication, which removes an editorial-bias path and equally removes human review | PARTIAL | not planned |
| Timely | Digests are generated four times a day per theater and track; lag against the expert benchmark is measured, not asserted | BUILT | |
| Based on all available sources | Multi-adapter ingestion — RSS, GDELT, Telegram web preview, Telegram MTProto, X — into one hash-deduplicated document store, with a per-theater corpus and a source and platform mix cap so one loud platform cannot fill a batch | PARTIAL | not planned |
| Properly describes the quality and credibility of underlying sources | Every source carries a citation profile and a five-value hedging distribution per theater, the per-claim evidence panel shows the documents behind a claim, and a generated narrative descriptor states the platform, the citation volume and date span in the named reference corpus, the hedging distribution and the registry status; no credibility level is asserted | PARTIAL | planned |
| Properly expresses and explains uncertainty | Every rendered claim carries an ICD 203 likelihood band with its published percentage range beside a separately labelled corroboration-derived confidence level, both computed by a signed, versioned mapping from the source's own estimative posture and the independence of the documents behind the claim; the sub-even bands and the highest band are never machine-assigned, because there is no refutation mechanism and no analyst-verified tier yet | PARTIAL | not planned |
| Distinguishes underlying information from assumptions and judgments | A claim cannot be committed without at least one source document — a deferrable database trigger fails the transaction otherwise — and a deterministic publication guard rebuilds or drops any event whose prose would state a single-document disputed allegation as fact | BUILT | |
| Incorporates analysis of alternatives | Divergences against the expert benchmark are recorded per run and shown on the scoreboard, but no structured alternative-hypothesis method is implemented | GAP | not planned |
| Demonstrates customer relevance and addresses implications | Products are scoped per theater and per track so a reader selects the lens rather than filtering a firehose; no explicit implications section is generated | PARTIAL | not planned |
| Uses clear and logical argumentation | Every rendered claim is a discrete assertion with its own evidence set, hedging label, and copyable citation, rather than an unattributed narrative | PARTIAL | planned |
| Explains change to or consistency of analytic judgments | Digests are archived per date and are addressable, so change is inspectable, but no explicit change statement is generated between editions | GAP | not planned |
| Makes accurate judgments and assessments | Not claimed. What is measured and published is coverage against a named expert benchmark, with the non-independence caveat carried beside the number | PARTIAL | not planned |
| Incorporates effective visual information | Hedging-mix bars, per-year citation histograms and scoreboard tables; no map or geospatial rendering | PARTIAL | not planned |
ICD 206 — Sourcing Requirements for Disseminated Analytic Products (January 22, 2015)
| Requirement | What BNOW does | Status | Roadmap |
|---|---|---|---|
| Source Reference Citation for every disseminated claim | A deferrable constraint trigger raises and fails the whole transaction if any claim commits without at least one source-document link; the application layer writes the claim and its links in one transaction; a migration test guards the trigger against regeneration | BUILT | |
| Source descriptors conveying reliability, bias and limitations | A deterministic template generates a descriptor per source and per reference corpus — platform, citation volume and span, the hedging distribution as shares in the corpus's own vocabulary, registry status, and the caveat that this describes how the benchmark cited the source rather than an independent audit — versioned and labelled as generated, with a pooled platform identity refused a profile and told so. Reliability and limitations are conveyed; the source's bias is not modelled | PARTIAL | not planned |
| Source summary statement for the product as a whole | A summary statement is generated per digest at read time from the claim-to-document join: distinct documents, channels and platforms, the share of claims resting on two or more of each, the sources supporting the most claims, the count resting on a single unconfirmed document, and the platform and transport mix against the batch cap the digest itself recorded — or an explicit statement that the digest recorded no mix figures | BUILT | |
| Sources preserved and retrievable for the life of the product | Every document is stored with its URL, body, fetch timestamp and content hash; no production code path deletes one, and a cited document cannot be removed at all because the citation link's foreign key declines to cascade — the property is now a written policy and a test that fails if it lapses | BUILT |
ICS 206-01 — Citation and Reference for Publicly Available Information, Commercially Available Information, and Open Source Intelligence (December 2, 2024)
| Requirement | What BNOW does | Status | Roadmap |
|---|---|---|---|
| Citation elements per source: author, title, URL, publication date, access date, source type | URL, title, publication date, adapter and platform are held and plumbed to every claim's evidence payload; the access date is retained but deliberately not presented, and no single standard-conformant citation string is emitted | PARTIAL | planned |
| A brief narrative quality descriptor per source | Generated per source and per reference corpus from citation volume and hedging distribution, rendered on the source detail surface and beside the digest the source supports, and never a letter grade or a headline score | BUILT | |
| Disclosure of AI or ML tooling: system name, model version, relevant parameters | Every extraction row carries a versioned extractor stamp, every synthesized digest carries a provider tag and a durable dispatch identity, and the routing registry version is recorded with it — none of which reaches a reader or the clipboard | PARTIAL | planned |
| Consistent PAI / CAI / OSINT vocabulary | Every live adapter ingests publicly available information; the distinction is not yet stated in product vocabulary or in any citation output | GAP | planned |
| Dynamic sources preserved at least one year from product issuance | Stated as policy: source documents are retained indefinitely and never for less than one year from any product citing them, with the four honest caveats — regeneration replaces claim rows, the content hash covers a prefix, a Telegram capture is a snapshot of the preview, and no third-party archival snapshot is taken | BUILT |
ICD 208 — Maximizing the Utility of Analytic Products (January 9, 2017)
| Requirement | What BNOW does | Status | Roadmap |
|---|---|---|---|
| Key judgments first — BLUF ordering | Digests lead with events and their supporting claims; the validation scoreboard leads with the benchmark result | BUILT | |
| Consistent, predictable product structure | One product shape per theater and track, dated, archived and addressable at a stable URL | BUILT | |
| Output a customer can reuse in their own product | Per-claim copy actions exist for report, link, evidence and text; no standard-conformant citation mode and no public API | PARTIAL | planned |
4. Two things we do that the standards do not require
Citation is enforced by the database, not by review. ICD 206 requires a source reference citation. It does not require that the storage layer make an uncited claim impossible. Ours does: a deferrable constraint trigger on the claims table raises an exception and rolls the entire transaction back if a claim reaches commit without at least one link to a stored source document. It is deferred to commit time, so a legitimate write that stores a claim and its evidence together passes, while any path that writes a claim and stops fails. It is re-asserted by the last migration applied, so a future schema regeneration cannot silently drop it, and a test fails the build if it is removed.
An external benchmark loop scores our product against a named expert publication. None of the four issuances requires a producer to publish a running comparison of its own output against an outside expert product. Each day, BNOW's digests are compared against the same-day Institute for the Study of War assessment, and the result is recorded as coverage, unsupported-claim rate, timeliness in hours, and an itemized list of divergences that distinguishes agreement, disagreement, benchmark-only items, and BNOW-only items. The results are public on the validation scoreboard.
5. Where the machine is disclosed as a machine
Every analytic artifact BNOW publishes is produced by an automated pipeline, and the stamps recording which machine produced it already exist in the data:
- Every extracted row carries a versioned extractor stamp derived from the model identity, the extraction prompt for that theater and track, and the content budget. It is a versioning contract, not a label: a change to any input is a corpus-wide event, and consumers filter to the current versions or they double-count.
- Every synthesized product carries the provider tag and the durable dispatch identity of the exact configuration each synthesis vote used.
- That identity includes the routing-approval registry version, so a published row can be traced back to the approval state that authorized a model to serve it.
None of this reaches a reader yet. Surfacing it in a copyable, standard-conformant citation is the planned work behind the PARTIAL rows under ICS 206-01.
6. What we deliberately withhold
Source text. No prose from the Institute for the Study of War and no full text from any ingested source appears in BNOW output. What we publish is source URLs, classifications, counts, dates, and scores. Source text may pass through an analysis step transiently; only the resulting classifications are stored and shown.
The reliability weighting itself.A source's reliability rating weights confirmed reporting above assessed, claimed, and unverified reporting. The numeric rating, the reliability-ranked ordering of the registry, and the exact weighting that would let the rating be reconstructed are an analyst privilege and are withheld from this page and from every non-privileged surface. Where a rating is shown at all, it is shown in context beside a cited source inside a digest, never as a standalone ranking.
One honesty note about that rating. When a source is cited without any hedging language at all, we hold it at the middle of the scale rather than forcing it into one of the explicit categories, because forcing it would corrupt the signal. That is a deliberate default, not a measurement — one of several reasons we do not present the rating as calibrated.
7. What we do not claim
We do not claim accuracy. What we measure and publish is coverage against a named expert benchmark — agreement with the same-day expert assessment. Agreement is not independent confirmation: the benchmark reads many of the same open sources BNOW reads, and that caveat is rendered beside the number, not in a footnote. We do not claim our source-reliability rating is calibrated, and it is not what produces the estimative labels beside a claim. Those labels — an ICD 203 likelihood band with its published percentage range, and a separately named corroboration-derived confidence level — are computed by a fixed, versioned mapping from two things only: the source's own hedging posture, and how independent the documents behind the claim are. They are not an analyst's judgment, and no human has reviewed them. Two limits follow, and they are deliberate: we never state that a claim is less likely than even odds, because we have no mechanism that refutes a claim; and we never state the highest band, which is reserved for a tier an analyst has verified. The row above stays marked partial for exactly those reasons.
8. Questions and corrections
If you believe a row above overclaims, or that a source or claim is misattributed, we want to hear it. Corrections and requests are handled through the process described in our Privacy Notice. For methodology review or partnership, request access.