Second measurements
3,226 of 4,472
GitHub security advisories published since April that took longer than the documented 72 hours to be reviewed; Dependabot does not alert until review, and the median wait was 8 days
SM-021 · GitHub Advisory Database · published
1,202,449 of 2,125,343
Federal grant sub-awards reported after the 2 CFR 170 deadline at or above the reporting threshold since October 2017; GAO found 41% of non-pandemic sub-awards late through 2023, state agencies carry three-quarters of ours
SM-019 · GAO, Education, GSA & three states · questions sent
20 of 30
Brazilian RPKI members whose lapsed manifests were deleted within five days; validators had dropped their route authorisations in the first hour
SM-014 · registro.br & RIPE NCC · RIPE NCC replied
1,816 of 13,871
CVE records that reached the CVE List more than three days after the numbering authority’s own public date, against a 72-hour MUST; ZDI, Dell, Drupal and Arista late on 1,107 of 1,680
SM-018 · CVE Program & four CNAs · sent, awaiting reply
9 seals from 8 organisations, each for answering on the record or fixing what we measured. Round trips.
Know a public claim that doesn’t add up? A log that says “every release”, a whitepaper that promises a schedule, a dataset everyone cites. Send it: hello@markovianprotocol.com. We read every tip, and we only take on claims that public data can settle.
Papers
| ID | Measurement | Result | Status |
|---|---|---|---|
| SM-021 | GitHub advisory review: docs say up to 72 hours; since April, 72.1% of reviewed advisories took longer 2026-10-19 · leaf 11365 · DOI 10.5281/zenodo.23439042 | 3,226 of 4,472 advisories published since April took longer than 72 hours to be reviewed (72.1%), median 8 days; January to March, 4.6%. Dependabot does not alert until review. | Sent 19 Oct, awaiting reply |
| SM-019 | Federal sub-awards reported late: GAO found 41% of non-pandemic, ours say 56.6%, state agencies carry three-quarters 2026-10-13 · leaf 11045 · DOI 10.5281/zenodo.23330729 | 1,202,449 of 2,125,343 sub-awards at or above the threshold late (56.6%), $809.5B net; state governments carry 76.9% of the late reports. Corrected 13 Oct. | Sent 13 Oct to six agencies, awaiting reply |
| SM-018 | CNA 72-hour rule: four CNAs late on two records in three 2026-10-11 · leaf 11044 · DOI 10.5281/zenodo.23305970 | 1,816 of 13,871 records late (13.1%). ZDI 77%, Dell 49%, Drupal 82%, Arista 86%; Microsoft 0.5%. | Sent to four CNAs 12 Oct and the CVE Secretariat 15 Oct, awaiting reply |
| SM-017 | FCC public-file lists: one in ten late in 2025 2026-10-14 · leaf 11043 · DOI 10.5281/zenodo.23374752 | 6,177 of 59,982 lists late in 2025 across 18,041 stations; 3,329 over a month late. | Sent 14 Oct, awaiting reply |
| SM-016 | Toxic-release reports: a thousand plants a year late, 61 EPA cases 2026-10-03 · leaf 11042 · DOI 10.5281/zenodo.23130553 | 5,964 of 473,709 forms over a year late; EPA actions fell from 52 a year to 16. | Sent 4 Oct, awaiting reply |
| SM-014 | RPKI: Brazil gives a member five days to fix a lapsed manifest; validators give zero 2026-10-03 and 2026-10-08 · leaf 11040 · DOI 10.5281/zenodo.23319187 | 30 members lapsed in the week to 3 October; by 8 October 20 were deleted, 10 recovered. Validators drop a lapsed member’s authorisations in the first hour; RIPE’s service still carried 34 lapsed points under 18 CAs. | RIPE NCC replied 14 Oct; NIC.br awaited |
| SM-015 | Tiled CT logs take seconds, old logs an hour, browser lists lag both 2026-10-03 · leaf 11041 · DOI 10.5281/zenodo.23123269 | 43 tiled logs under 6 s; Cloudflare Nimbus 61 min; Apple’s list calls 12 closed shards usable. | Apple fixed its list 15 Oct; Chrome awaited |
| SM-012 | CA audits: one in eleven late; Windows trusts a root audited in 2019 2026-10-03 · leaf 11039 · DOI 10.5281/zenodo.23123265 | 11 of 119 audits past 92 days, 7 missing; Microsoft trusts 24 roots audited over 15 months ago. | No reply; filed in Bugzilla 16 Oct |
| SM-011 | Firefox’s revocation list is months behind the logs 2026-10-02 · leaf 11038 · DOI 10.5281/zenodo.23123261 | Reader 112 days behind; 68 of the top 1,000 sites not covered, mozilla.org among them. | Replied 5 Oct, 4 of 4 |
| SM-010 | Hugging Face says every file goes through its malware scanner. Nothing over 2 GB does, and the badge says safe anyway 2026-10-02 · leaf 11329 · DOI 10.5281/zenodo.23123005 | 2,539 of 2,551 files under 2 GiB scanned; 0 of 487 at 2 GiB or more. 476 of those show “safe”. | Hugging Face fixed the docs 6 Oct; badge question open |
| SM-009 | Firefox’s revocation list misses a third of day-one revocations 2026-10-02 · leaf 11328 · DOI 10.5281/zenodo.23123001 | 37 of 108 certificates revoked within 24 hours missing, for up to 28 days. | Closed 17 Oct |
| SM-008 | A federal directive says CISA rates every CVE. Since March, it has rated almost no Linux kernel bugs 2026-10-02 · leaf 11327 · DOI 10.5281/zenodo.23122997 | 1,731 of 1,731 exploited CVEs rated. 1 of 158 sampled kernel CVEs since March. | Sent 2 Oct, six questions open |
| SM-007 | Docker’s attestations arrived unsigned 2026-10-02 · leaf 11326 · DOI 10.5281/zenodo.23122993 | 7,368 of 7,372 Linux images carry attestations. 0 of 45 sampled are signed. | No reply; data posted on Docker’s roadmap issue 16 Oct |
| SM-006 | x402 payments on Base fell three-quarters in two months 2026-08-05 and 2026-10-01 · leaf 11325 · DOI 10.5281/zenodo.23122988 | 234,490 a day in August, 63,003 on 1 October; two wallet groups send 80%. | Sent 1 Oct, awaiting reply |
| SM-005 | Meta’s private-AI log refreshes every 6 hours, not 3 2026-09-30 · leaf 11324 · DOI 10.5281/zenodo.23122986 | Lists every 6 hours, not 3. New software entries in 37 of 70 weeks. | Sent 1 Oct, awaiting reply |
| SM-004 | whisper.online’s ledger passed every check 2026-09-12 to 2026-09-30 · leaf 11323 · DOI 10.5281/zenodo.23071390 | Every check passed; a fix cut duplicate updates from ten per change to one. | Confirmed by operator |
| SM-003 | Wild SBOMs: nearly half from one robot 2026-09-30 · leaf 11322 · DOI 10.5281/zenodo.23123301 | Nearly half came from one automated run in 2022. | One author replied; no correction yet |
| SM-002 | Claude Code dataset sees 1 in 11 of its pull requests 2026-09-30 · leaf 11321 · DOI 10.5281/zenodo.23123115 | Its search catches about 1 in 11, and an unusual slice of them. | Hao Li answered 16 Oct: v5 uses the wider query |
| SM-001 | Google’s Pixel ledger lost a month, for eight months 2026-09-15 · rechecked 2026-09-30 · leaf 11320 · DOI 10.5281/zenodo.23070510 | January 2026 was missing; Google added it, Android 17 backfill in progress. | Confirmed, fixed |
How we work
One sentence an organisation published about itself, recomputed by a path it doesn’t control, then rechecked on a schedule until it ends. The full method and the shape of the thing are on their own page; the claims we have found and not yet measured are on the backlog; every promise, measured or listed, with its state, clock and ending, is one row on the registry; and every version of the lists our papers read, Chrome’s and Apple’s CT log lists and the CCADB export, is pinned in our log on the list history. New papers, replies and fixes: RSS feed. Every paper and null result as one table: the dataset on Hugging Face, or search and filter it in your browser with Datasette Lite, nothing to install.
- Quote the claim. Every check starts from the organisation’s own words, copied from its own page, saved and hashed.
- Name the kill condition first. Before measuring, we write down what result would mean there is no story, and stop if we hit it.
- Recompute independently. Our own code, written from the public spec, against public data. Every script is published.
- Rerun before publishing. A finding has to hold on a second, separate run.
- The subject hears from us directly, with specific questions: maintainers of open-source projects get a copy seven days before publication; everyone else on the day it publishes. Their reply is printed in full and scored.
- No positions, no payments. We hold no stake in anyone we measure, take no money for any measurement, and share nothing before publication with anyone but an open-source maintainer who gets the advance copy and, where a paper names one, its reviewer.
- Corrections stay visible. If we got something wrong, the page says so and the old version stays in the log.
- The clock runs both ways. Thirty days without a reply, and the status says so in red, counting. Reply, fix it, and let the recheck confirm it, and the paper carries the time you took in green and a seal you can keep.
Open clocks
Every party we asked, the day the note went, and where its 30-day clock stands today (2026-10-19). A reply is printed on the paper and scored; silence past day 30 turns red and counts. The row then goes to whoever can act on the finding.
| Paper | Asked | Sent | Clock | Ending |
|---|---|---|---|---|
| SM-002 | the AIDev authors | 2026-09-30 | 11 days left | clock ends 2026-10-30 |
| SM-003 | the authors | 2026-09-30 | 11 days left | clock ends 2026-10-30 |
| SM-005 | Meta | 2026-10-01 | 12 days left | clock ends 2026-10-31 |
| SM-006 | the authors | 2026-10-01 | 12 days left | clock ends 2026-10-31 |
| SM-007 | Docker | 2026-10-02 | 13 days left | clock ends 2026-11-01 |
| SM-008 | CISA and the kernel CVE team | 2026-10-02 | 13 days left | clock ends 2026-11-01 |
| SM-010 | Hugging Face | 2026-10-02 | 13 days left | clock ends 2026-11-01 |
| SM-012 | the CCADB steering committee and Microsoft's root program | 2026-10-03 | 14 days left | clock ends 2026-11-02 |
| SM-015 | Apple's and Chrome's CT programs | 2026-10-03 | 14 days left | clock ends 2026-11-02 |
| SM-016 | EPA's TRI Program | 2026-10-04 | 15 days left | clock ends 2026-11-03 |
| SM-014 | NIC.br (registro.br) and the RIPE NCC RPKI team | 2026-10-12 | 23 days left | clock ends 2026-11-11 |
| SM-018 | ZDI, Dell, Drupal and Arista | 2026-10-12 | 23 days left | clock ends 2026-11-11 |
| SM-019 | California, Texas and Michigan education agencies, Dept. of Education, GSA, GAO | 2026-10-13 | 24 days left | clock ends 2026-11-12 |
| SM-017 | the FCC ([email protected]) | 2026-10-14 | 25 days left | clock ends 2026-11-13 |
| SM-018 | the CVE Program Secretariat (support form, CAN-2026-2039953) | 2026-10-15 | 26 days left | clock ends 2026-11-14 |
| SM-021 | GitHub Advisory Database | 2026-10-19 | 30 days left | clock ends 2026-11-18 |
| SM-001 | Google, Pixel Binary Transparency | 2026-09-15 | replied in 7 days | fixed in 7 days; daily recheck at 0 missing |
| SM-004 | whisper.online | 2026-09-18 | replied the same day | fixed in 4 hours |
| SM-009 | Mozilla, CRLite | 2026-10-02 | replied in 3 days | answered in 3 days; the 17 October full filter shipped on the date given and cleared the gap (0 of 401 good on 18 October) |
| SM-011 | Mozilla, CRLite | 2026-10-02 | replied in 3 days | answered in 3 days |
Checked, nothing found
Claims recomputed by a separate path that held. Each was re-run on the date shown. How close each came.
| Date | Claim | Check | Result |
|---|---|---|---|
| 2026-09-30 | Every entry in the Armored Witness production firmware log names the source it was built from | Each of the 28 entries' commit_fingerprint compared with the commit its tag points to on GitHub (script) | 27 of 28 match the v-prefixed release tag. The recovery image's commit is on usbarmory/armory-ums master, but no 0.1.0 tag exists there or in any fork. |
| 2026-09-30 | Cloudflare's Plexi auditor has verified every WhatsApp key-transparency (v2) epoch from 722,677 to its reported last verified epoch | 150 epochs drawn uniformly from 722,677 to 1,297,263, each audit fetched and its epoch, namespace and digest checked (script). Audit signatures not checked here. | 150 of 150 retrievable and consistent. |
| 2026-10-01 | A Large-Scale Dataset of MCP Implementations on GitHub (MSR '26) describes real-world MCP development | Their released data and code, and GitHub's counts of matching repositories above and below the code's 50-star minimum (script) | Every repository has at least 50 stars, a cut the paper doesn't state; that is 4.3% of matching repositories. Python and TypeScript lead on both sides of it, and no owner holds more than 1%. |
| 2026-10-01 | PyPI: “17% of all uploads to PyPI in the last year included an attestation”, and more than 20% came via Trusted Publishing (2025 in review) | 1,000 file uploads drawn from PyPI’s own changelog across 2025 (50 random three-day stretches, 20 files each), each looked up in PyPI’s Integrity API (script) | 17.9% of uploads carry an attestation (95%: 15.1–20.8%). Of the files still on PyPI, 20.5%, which is also a floor on Trusted Publishing. 178 of 179 come from GitHub. |
| 2026-10-01 | Same Name, Different Server (arXiv 2609.14119): 4.2% of multi-version servers in the official MCP registry moved their endpoint to a different host while keeping their name, which the registry “cannot distinguish” from a takeover | Every version in the registry from its public API, comparing each server’s endpoint hosts between consecutive versions (script) | 418 of 10,402 (4.02%) to the end of August. Of the 135 whose name is tied to a verified domain, 116 stayed with that domain’s owner. 283 of the 418 are named after a GitHub account, where the registry checks no domain at all. |
| 2026-10-01 | Never Emitted (arXiv 2609.33099): GitHub shows who reported each vulnerability on its advisory but leaves that credit out of the CVE record and its OSV file | The newest 150 GitHub-assigned CVEs whose advisory credits someone, each CVE record read from CVE Services and each OSV file from GitHub’s advisory database (script) | 0 of 150 CVE records and 0 of 150 OSV files carry credits; all 150 CVE records carry metrics and problemTypes. |
| 2026-10-02 | Homebrew: homebrew-core uses Sigstore “to cryptographically attest to all bottles built in the official Homebrew CI” (2024) | 600 bottle files drawn at random from formulae.brew.sh in two independent samples, each looked up in GitHub’s attestation API under homebrew-core’s CI and Homebrew’s backfill signer (script) | 599 of 600 attested; every CI attestation carries a Sigstore transparency-log entry. The exception: ht 2.1.0’s two Monterey bottles, built 24 April 2024 and still served, which neither signer attests. Reported as homebrew-core#314985. |
| 2026-10-02 | NIST NVD: for CVEs in CISA’s known-exploited catalogue, “our goal is to enrich these within one business day of receipt” (from 15 April 2026) | Every one of the 163 catalogue entries added since 15 April, timed from the later of catalogue addition and NVD publication to NIST’s first analysis in NVD’s own change history (script) | 163 of 163 within one business day. 68 had been analysed before they reached the catalogue; the other 95 took a median of 19.5 hours. |
| 2026-10-02 | Kubernetes: “The Kubernetes release process signs all binary artifacts (tarballs, SPDX files, standalone binaries) by using cosign’s keyless signing.” | Every standalone binary in the latest patch of each minor version from 1.26 to 1.37 (408 files) checked for its .sig and .cert, and one kubectl signature per version verified with our own code (presence, verification) | 408 of 408 carry both; 12 of 12 signatures verify, all from the release identity Kubernetes documents. Certificate chain and transparency-log entries not checked here. |
| 2026-10-02 | Proton Mail key transparency: “Normally a ProtonKT epoch should be published every 4 hours. The maximum publishing interval is 72 hours” (whitepaper, p. 14) | Every epoch certificate Proton logged to Certificate Transparency since July, found through Cert Spotter; each certificate’s name encodes its epoch number and issue time (script) | 515 epochs, 4 July to 2 October: median interval 4.00 hours; longest 38.1 hours (27–28 September), inside the 72-hour limit. Two epoch numbers at the start of the window aren’t visible, probably expired from the search index. |
| 2026-10-03 | CCADB incident-report rules: open reports “MUST be updated: on or before the ‘Next update’ date in the Whiteboard field”, and CA Owners “MUST respond within 7 days” to comments and questions (ccadb.org/cas/incident-report) | Every open bug in Bugzilla’s CA Program tagged [ca-compliance] (92 on 3 October): the Whiteboard’s Next-update date, the assignee, and every comment with its author and time; run twice the same day (script, results) | 24 reports carry a Next-update date; none is past it without a CA update. No question from a root program or the community has waited more than 7 days for a CA reply. Two flags on first pass were an administrative note and a community thread with a scheduled date. The rule polices itself: one open report is Microsoft PKI Services filing an incident for having missed its own seven-day update on another. |
| 2026-10-03 | CA/Browser Forum Baseline Requirements 4.9.7: a CA issuing subscriber certificates “MUST update and publish a new CRL at least every seven (7) days”; the CRL profile caps nextUpdate at “at most 10 days after the thisUpdate” for CRLs covering subscriber certificates | Every CRL URL the CCADB lists for its 2,206 not-revoked TLS-capable intermediates (16,141 distinct URLs from 62 CA owners), fetched and parsed: thisUpdate, nextUpdate, issuer, entry count; a 1,500-URL sample fetched again an hour later (script, findings) | 15,982 CRLs covering subscriber certificates: median 3 hours old, 99% under a day; median validity 7.0 days, 90th percentile 9.96 days against a 10-day cap. Not one TLS CRL is stale or expired. The five over seven days old are timestamping, qualified-signature and legal CAs the CCADB marks with a server-authentication bit, plus one real one: Brazil’s ITI, trusted by Microsoft alone, serves an EV CRL whose nextUpdate was 24 March 2023, 54 entries, three and a half years stale. |
| 2026-10-03 | GitHub: “Public repositories that generate artifact attestations use the Sigstore Public Good Instance. A copy of the generated Sigstore bundle is stored with GitHub and is also written to an immutable transparency log that is publicly readable on the internet.” | The latest release of each of the 1,000 most-starred repositories pushed since September; every attestation GitHub holds for their assets (388 bundles from 63 repositories), each bundle’s transparency-log entry looked up on the public Rekor instance by index (collect, verify) | Every Actions-generated attestation is in the log: 161 of 161 found on Rekor at the stated index. The other 227 bundles are GitHub’s own release attestations, which come with immutable releases: signed by GitHub’s certificate authority with a one-year certificate, timestamped by GitHub, and in no public log. GitHub’s release documentation doesn’t claim one; the sentence above is about the first kind. A verifier of an immutable release is trusting GitHub alone. |
| 2026-10-03 | CVE Program, CNA Operational Rules 4.5.1.4: a CNA “MUST publish a CVE Record to the CVE List within 72 hours of Publicly Disclosing a CVE ID” | The newest 780 reviewed GitHub Security Advisories that carry a CVE, each CVE record read from CVE Services; lag is the record’s datePublished minus the advisory’s published time (script, results) | For GitHub’s own CNA, 336 of 336 records were published before the advisory that discloses them, by a median of 26 hours, so the rule holds there by construction. The same is true of VulnCheck (288), OpenJS (84) and HeroDevs (24). The rule is only measurable where a CNA announces first and files later. |
| 2026-10-03 | SEC Rule 16a-3(g): an insider’s Form 4 “must be filed before the end of the second business day following the day on which the subject transaction has been executed”; Regulation S-K Item 405: the proxy must name anyone who filed late | Every Form 4 in the latest fiscal year of 498 S&P 500 issuers from EDGAR’s submissions feed (34,490 forms), business days from transaction to filing; the Section 16(a) paragraph of each issuer’s proxy; every form more than five business days late read in full and checked against the proxy (scripts, results) | 2,531 forms filed after day two, 279 after day five, across 116 issuers. 188 of the 214 proxies with the section name their late filers, down to a five-share trade. Reading the 279: most of the rest are exempt transfers, plan purchases or inheritances. Two omissions survive: an AutoZone director’s $1.08M sale reported 303 business days late with a proxy that carries no section, and a First Solar controller’s 560-share vest reported 122 days late, likewise. AutoZone’s next proxy is due in late October. |
| 2026-10-04 | Charity Commission for England and Wales, annual report 2024–25: “92% of charities were up to date with filing at the end of 2024-25. This represents a significant recovery from 2023-24 (81%)” | The Commission’s own register extract of 4 October 2026 (annual-return history: due date, date accounts received, income) for every registered main charity with income of £25,000 or more, by filing cycle (script, results) | Received by the due date: 81.8% of 67,689 for the 2023 cycle, 91.4% of 71,433 for 2024, 92.6% of the 52,663 due so far for 2025. The Commission’s numbers hold. By income the on-time share is 95%; the late ones are late by a median 45 days, and 408 charities with income over £1 million filed late for 2024, the British Council (56 days) and the Salvation Army (112) among them. The extract carries no row for a return never filed, so non-filers are invisible here. |
| 2026-10-02 | Every Certificate Transparency log promises a maximum merge delay: 24 hours for the classic logs, 60 seconds for the tiled ones (the mmd field of the Google and Apple log lists; RFC 6962 §3) | A fresh certificate submitted to each of the 64 usable or qualified logs that will take a current certificate (our own chain, SM-009’s certificates with their CCADB intermediates, and entries harvested from 2027h2 and 2028 tiled logs), each log then polled on its own thread until it served the entry: an inclusion proof for classic logs, a checkpoint past the SCT’s leaf index for tiled ones (script, harvester) | 64 of 64 inside the promise. Tiled: all 43 within 6 seconds, most within 1. Classic: DigiCert 10 seconds, Google and Sectigo 1 to 2 minutes, TrustAsia 2.5 minutes, Cloudflare’s Nimbus 46 and 61 minutes, the only logs that publish a new tree head less often than every minute. The 24-hour figure Firefox’s revocation filter waits on (SM-009) is a thousand times what the slowest Google log took. |
| 2026-10-02 | Debian release team: “Since yesterday, we have enabled our migration software to block migration of new packages that can’t be reproduced or existing packages (in testing) that regress in reproducibility” (debian-devel-announce, 10 May 2026) | Every binary in testing with a rebuild verdict on reproduce.debian.net (36,003), split by what changed since the gate, scored with britney’s own per-binary regression rule, and the remainder crossed with the release team’s public hints and each package’s migration date (script) | 457 are unreproducible. 128 were there before the gate; 231 replaced a version that was already unreproducible, which the rule allows; 76 carry a public release-team hint; 10 got their verdict after crossing. The last 12, all petsc, are the gate working: britney flagged them in June, the release manager judged them renamed rather than regressed, and waved them through on the record in #1135890. 0 unexplained. |