THE WORLD IS NOT STANDING STILL.RSS
BIG CHANGE.
BIG CHANGE, one tap away
BIG CHANGE, one tap away

Make room for the big picture.

Add BIG CHANGE to your home screen or desktop. It opens in its own window, with the latest stories when you’re online.

iPhone & iPad
Open bigchange.ai in Safari. Tap Share, then Add to Home Screen. Turn on Open as Web App if shown, then tap Add.
Android
Open bigchange.ai in Chrome. In the browser menu, choose Install app or Add to Home screen.
Desktop
In Chrome or Edge, use the install icon in the address bar or the browser’s app menu. In Safari on Mac, choose File → Add to Dock.

Markdown edition

# Real PubMed IDs can still point to the wrong evidence

> A preprint found most cited PubMed IDs existed while few were judged relevant.

By BIGCHANGE.AI Editorial

Published: 2026-10-25T13:44:39.985Z
Updated: 2026-10-25T13:44:39.985Z
Canonical: https://bigchange.ai/blog/real-pubmed-ids-wrong-evidence-biomedical-agents

![Journal shelves with one orange-marked volume.](https://bigchange.ai/api/media/file/pubmed-real-id-evidence-check-hero-v2.png)
AI-generated conceptual illustration by BIG CHANGE.

An October 21 preprint found that 327 of 335 PubMed IDs supplied by closed-book agents resolved to real records. A separate language-model judge rated only three of those 327 relevant to the biomedical claims.

Author Mohammed Saqlain compared closed-book answering, fixed-query PubMed retrieval, agent-written queries and an oracle given the correct abstracts. Each ran on 100 sampled SciFact development claims with two Gemini Flash Lite models. Agent search reached 62% verdict accuracy with both; its advantage over other realistic setups was not statistically significant. Oracle accuracy reached 79% and 86%.

## The big change

A citation existence check leaves relevance untested. The relevance finding rests on one automated judge checked against 50 controls. This small, single-provider study does not establish clinical reliability.

## Sources

- [Saqlain's preprint](https://arxiv.org/html/2610.29812v1) — Methods, citation audit and statistical limits.