Markdown edition

# Safety results were public at launch for 9 of 857 releases in one AI model audit

> SemiAnalysis found model-specific safety results available by launch for nine of 857 releases by nine China-based developers. The audit measures public disclosure, not private testing or model safety.

By BIG CHANGE Editorial

Published: 2026-10-11T22:00:10.466Z
Updated: 2026-10-11T22:00:10.466Z
Canonical: https://bigchange.ai/blog/ai-model-safety-results-public-at-launch-semianalysis

![Conceptual charcoal illustration of a glass-fronted public notice case with generic papers and one orange-cornered sheet beside another notice.](https://bigchange.ai/api/media/file/china-safety-disclosure-hero-v2.png)
A conceptual public notice case illustrates the difference between a release notice and a publicly visible result. It does not depict actual developer documents or scale the audit's 857 releases; a missing public result does not establish whether private testing occurred. AI-generated by BIG CHANGE.

When a new AI model is released, a claim that it was "evaluated" tells users little about what the tests found. In an [October 8 analysis](https://newsletter.semianalysis.com/p/beijing-will-not-pace-the-frontier), SemiAnalysis counted 857 releases by nine China-based developers from 2021 through September 15, 2026. It found a published safety result for 31 releases (3.6%). For just nine (1.1%), a result was available at or before the model's release.

The figures describe what the researchers could find in public developer materials. They do not show how many models were tested privately or whether any particular model is safe.

## The big change

- **What changed:** SemiAnalysis has put a number on a documentation gap across a defined set of releases: nine of 857 had a model-specific safety result people could read by launch day.
- **Why it matters:** Someone assessing a model at release needs results for that version. A generic assurance of testing, or a result published after release, cannot answer the same question at that moment.
- **What to watch:** For subsequent releases, the useful signal is a dated result tied to the exact model and available when users first get access. The audit's disclosure rate can change as documents appear, but later publication does not change what was available on launch day.

## What the audit counted

SemiAnalysis says its census covered 741 product-model releases and 116 research-model releases from Alibaba, ByteDance, Tencent, Baidu, DeepSeek, Moonshot, Zhipu (Z.ai), MiniMax and StepFun. It gave each release a first-public date, then checked developer model cards, release notes and technical reports for a safety result it could match to that release.

A qualifying result had to give a quantitative or substantive finding about a named model. The subjects included harmful output, resistance to jailbreaks, toxicity, privacy, refusal behavior and dangerous capabilities. A statement that a model was safety-trained or evaluated, without a finding, did not qualify. Nor did the researchers extend a result for one flagship model to other sizes or later snapshots.

That strict matching rule makes the denominator important. The 857 entries are counted releases, including variants, rather than 857 equally capable frontier systems. SemiAnalysis notes that developers name variants at different levels of detail. Its count is therefore a way to examine documentation across this set of releases, not a sound basis for ranking individual companies.

## When results appeared

Of the 31 releases with a qualifying published result, nine had one at or before release. Sixteen were documented later; SemiAnalysis reports a median delay of 42 days for that group. For the other six, the researchers could not establish the timing or the match to a specific model. Those six belong in the 31 with some published result, but they cannot be counted as launch-day disclosures.

The remaining cases also need careful wording. SemiAnalysis found 10 releases with a claim of evaluation but no figures, and three known only through press or investor accounts for which it could not retrieve developer documentation. It found no safety disclosure for 813 releases in the materials it checked. Together, those groups and the 31 with results make up the 857 releases.

The [Reuters report on the analysis](https://uk.marketscreener.com/news/china-ai-developers-publish-safety-tests-for-just-3-6-of-model-releases-report-finds-ce785ddcd98cf623) also stresses that companies may have performed private tests. "No disclosure found" is a statement about the public record examined here. It cannot establish that testing did not happen, and a published test does not by itself establish the model's underlying safety. SemiAnalysis did not provide a comparable release-level rate for US developers in this audit.

For a reader choosing or assessing a model, the practical distinction is simple: identify the exact version, look for a dated, model-specific result, and check whether that result was available when the model was released. That evidence says what the developer made visible. It is only one part of evaluating a model for a particular use.

## Sources & further reading

- [SemiAnalysis, "Beijing Will Not Pace the Frontier: China's Speed-First AI Safety Regime" (October 8, 2026)](https://newsletter.semianalysis.com/p/beijing-will-not-pace-the-frontier). The accessible methods and results passage states the nine-developer sample, 2021–September 15 window, 857-release denominator, document search, matching rule and timing groups. The page is marked paid; we did not independently inspect its underlying release-level dataset or reproduce its searches.
- [Reuters, "China AI developers publish safety tests for just 3.6% of model releases, report finds" (October 9, 2026), via MarketScreener](https://uk.marketscreener.com/news/china-ai-developers-publish-safety-tests-for-just-3-6-of-model-releases-report-finds-ce785ddcd98cf623). Reuters separately reports the main counts and the distinction between absent public disclosure and possible private testing. Its account is reporting on the SemiAnalysis analysis, not an independent replication of the dataset.

## Sources

- [SemiAnalysis, Beijing Will Not Pace the Frontier: China's Speed-First AI Safety Regime](https://newsletter.semianalysis.com/p/beijing-will-not-pace-the-frontier) — Primary methods and release-census passage accessible on a page marked paid. Gives sample, match criterion and timing categories. Release-level dataset and document searches not independently replicated.
- [Reuters, China AI developers publish safety tests for just 3.6% of model releases, report finds](https://uk.marketscreener.com/news/china-ai-developers-publish-safety-tests-for-just-3-6-of-model-releases-report-finds-ce785ddcd98cf623) — Reuters reporting on the SemiAnalysis audit, corroborating core reported counts and the private-testing caveat; not an independent replication.