Methodology for the BigSkyLabs Frontier AI Claim Tracker · Tracking since October 5, 2026
Current tracker snapshot
Counts update in real time as the detection pipeline runs. View all claims →
Most recent Marketing Hype flag
"Doximity's new dose verification tool outperformed general-use AI models"
The claim uses comparative language ("outperformed") without citing independent verification or peer-reviewed research. The article notes this is "research released by the clinical AI company" (i.e., self-reported), making it promotional rather than independently verified.
The Frontier AI Claim Tracker runs as part of the BigSkyLabs weekly intelligence pipeline, which processes 600+ articles per week from RSS feeds, Perplexity Research API, academic databases, and regulatory sources.
For each article, a large language model reviews the text and extracts specific factual claims made by frontier AI companies — OpenAI, Anthropic, Google, Amazon, Meta, Microsoft, NVIDIA, and others — about their own products, capabilities, or regulatory status. The model is instructed to focus on first-party assertions: statements the company made, not what analysts or journalists concluded about them.
Extracted claims are deduplicated against the existing record. A claim already in the tracker is not re-added; instead, new source articles are attached to the existing entry, strengthening or weakening the evidentiary basis and potentially triggering a verdict update.
The pipeline also runs on demand from the Admin Pipelines page, allowing manual sweeps when a high-volume event (a product launch, a congressional hearing, an FDA action) produces more claims than the daily cycle captures.
Verdicts are assigned at detection and updated when new evidence enters the record. A verdict is never permanent — the tracker reflects the best available evidence as of the date shown in each entry.
The claim is supported by verifiable, independently produced evidence available in the article or cited source at time of detection. This includes peer-reviewed clinical studies, published safety benchmarks, independently audited performance data, or formal regulatory approvals whose scope matches the language of the claim.
This is the highest bar. A company announcing a new capability without a link to the underlying study does not qualify — the evidence must be present in the source, not promised in a future publication.
The claim was made by the company but no independent, verifiable citation accompanied it at time of detection. This is the most common verdict — the majority of capability announcements and partnership claims from major AI companies are issued via press release or blog post without peer-reviewed backing.
Alleged/Unverified is not a finding of falsity. It is a finding of insufficient evidence. Many of these claims may be accurate. None of them have cleared an independent evidence standard. This verdict changes to EVIDENCE_BACKED when the underlying data is published and verified.
The claim uses superlative, absolute, or regulatory language in a manner that implies a scope, authorization, or performance level that the available evidence does not support. This includes claims that misrepresent the scope of a regulatory clearance, assert uniqueness ("first," "only," "most advanced") without citation, or use quantified performance figures that cannot be traced to a reproducible source.
Marketing Hype is the most consequential verdict in a healthcare context. A health system or clinician acting on a Marketing Hype claim — particularly a Regulatory Claim rated Marketing Hype — may make deployment or prescribing decisions based on an authorization the product does not hold.
One or more claims in the company's published evidence base directly contradict the original claim, or an independent peer-reviewed study finds materially different results from those the company stated. The contradicting evidence must be specific to the original claim — a general finding of AI limitations does not automatically contradict a specific capability claim.
As the wiki knowledge base grows, CONTRADICTED counts will rise. The tracker cross-references new incoming articles against existing claims. When a contradiction is detected, the original claim is flagged and a note is added linking both records.
The company has publicly walked back, corrected, or removed the original claim. Retracted claims remain in the tracker permanently. The original claim text, the original verdict, and the retraction source are all preserved. Deletion from the company's own materials does not remove the entry from this record.
Retraction is a significant event. When it happens, we publish a standalone note.
Each claim is classified by type. Type determines the evidence standard — Regulatory Claims are held to the highest bar; Capability Claims are the most common and the most frequently unverified.
The current verdict distribution — heavily weighted toward Alleged/Unverified, with isolated Marketing Hype flags — is not a static picture. It is a baseline. The tracker's analytical value grows non-linearly with the number of claims rated.
At 50 claims, per-company credibility patterns emerge: which companies issue the most unverifiable capability claims, which claim types generate the most Marketing Hype flags, and whether any company is consistently issuing claims that later get contradicted. These are patterns that don't exist at 25 claims but become statistically meaningful at 50.
At 100 claims, the tracker supports a credibility ranking — an ordered list of frontier AI companies by the percentage of their healthcare claims that have cleared an evidence bar. That ranking will be published. It will be citable. It will be uncomfortable for some companies.
A claim moves from Alleged/Unverified to Contradicted when a peer-reviewed study, an independent audit, or a regulatory finding directly contradicts what the company said. The first time this happens, it is a standalone story: a specific company made a specific claim about a specific healthcare capability, and the published evidence says otherwise. The claim, the original source, and the contradiction are all in the permanent record.
As the wiki knowledge base grows — cross-referencing new articles against existing claims — the Contradicted count will rise. The healthcare AI market is producing enough independent clinical evidence that contradictions are a matter of time, not possibility.
Retractions are rare but significant. When a frontier AI company corrects or removes a healthcare AI claim — whether under regulatory pressure, legal challenge, or internal review — that event is recorded here. The claim stays. The retraction source is added. The record does not follow the company's editorial decisions.
The infrastructure to hold healthcare AI claims accountable is being built in parallel. The FDA's Digital Health Center of Excellence is actively reviewing AI-generated clinical recommendations. The FTC issued AI claim enforcement guidance in 2024. The EU AI Act's Article 6 high-risk classification for clinical decision support is in force.
When a regulatory action or a class action lawsuit cites a company's capability claims, the tracker provides the dated, sourced record of what was claimed and when. That record is independent of what the company's website or press archive currently shows.
The tracker is designed to be cited — by journalists, by compliance officers, by plaintiffs' attorneys, and by regulators building the evidentiary record for enforcement actions. The value of any individual claim entry is low. The value of the full record at 500 claims, with a two-year history, is substantially higher — and it compounds.
Every post in the BigSkyLabs weekly intelligence series includes the current Marketing Hype count as a running tally. That number — currently 13 since tracking began — will grow over weeks and months as more product launches and regulatory announcements are rated.
When it reaches 10, we will publish a synthesis: the 10 healthcare AI claims most likely to mislead the health systems buying them. When it reaches 25, we will publish a pattern analysis: the specific language constructs that reliably predict a Marketing Hype verdict. The running count is not a footnote. It is the story.
If you believe a verdict is incorrect — because published evidence exists that was not captured at detection, because a claim was misattributed, or because the evidentiary standard was misapplied — you can submit a dispute.
Include: the claim ID (visible in the URL when you click a claim on the tracker), the proposed corrected verdict, and a link to the published evidence that supports the correction. Verdicts supported by peer-reviewed publications or formal regulatory documents will be reviewed and updated within 5 business days.
Submit a dispute → newsletter@bigskylabs.ai
Note: Requests to remove claims from the tracker without providing contradicting evidence are not accepted. Verified retractions result in a RETRACTED verdict, not deletion.