The NYT Says AI Doctors Are More Accurate—Time to Fire Your PCP? (Part 3 of 6)
My rebuttal to the NYT op-ed by Eric Topol, M.D., of Scripps Research and Pranav Rajpurkar, Ph.D., of Harvard University.
Dear Drs.
and :Let me respectfully push back on some of the implications of your The New York Times op-ed and the accompanying Substack article, both published on February 2, 2025. I strongly believe that when doctors have experience using AI, that combination should have synergy over AI alone. The studies that you cite, where AI allegedly outperformed doctors, must be scrutinized. Once you do, there’s usually something glaringly obvious that the authors (and the journal—and most definitely the mass media) overlooked.
Case in point: The Stanford JAMA study cited by you, which claimed 90% accuracy for AI alone versus 76% for Doctors + AI, was completely debunked by my rebuttal. The errors were so elementary that I have no idea how the 16 co-authors who signed their names to this paper—as well as the JAMA editors and reviewers—missed them all! And of course, the mass media picked it up and ran with the hype. This “AI outperforms doctors” hoax has gone way too far. It’s embarrassing that The New York Times and later CNN amplified it. They had 16 co-authors on the JAMA article, and apparently not a single statistician.
To their credit, the Stanford authors eventually backed off the New York Times claims—even though they were directly quoted in the article!
I haven’t had a chance to look at other papers in detail yet, but please stay vigilant. Even the peer review process isn’t perfect.
As of now, we can’t trust GenAI (e.g. Large Language Models, or LLMs) to perform independent tasks in medicine when it comes to the standard of care.
GenAI is built on transformer models, which, by the design of their loss function, will never produce a 100% accurate outcome. Instead, they generate a distribution of possible outcomes—leading to what is commonly referred to as hallucinations. There are other AI models outside of GenAI that may be better suited for precision medicine, but none of the studies you cite use those models.
Respectfully,
Sergei Polevikov
P.S. This is Part 3 of my 6-part series investigating how close we are to AI outperforming doctors. Here are the first two parts:
The “AI Outperforms Doctors” Claim Is False, Despite NYT Story - A Rebuttal (Part 2 of 6)
Don’t miss Part 4 of this series—a super deep dive and probably the most comprehensive breakdown of Why AI Will Replace Doctors—coming in the next few weeks. Bad news for doctors. Great news for engineers. Stay tuned.
Topol, whom I know, is a tool of the first order. You can assume, even without any analysis, that if he says something it is almost certainly wrong...usually massively so. So no surprises here. But thanks for keeping up the good work.