These 15 Health AI Companies Have Been Lying About What Their AI Can Do (Part 1 of 2)

The first health AI company in history is slammed with a lawsuit over 'deceptive claims' about its so-called 99.999% model accuracy.

Welcome to AI Health Uncut, a brutally honest newsletter on AI, innovation, and the state of the healthcare market. If you’d like to sign up to receive issues over email, you can do so here.

In this two-part article, I’m investigating the world of health AI companies that lied to clinicians and patients about their AI models’ accuracy—or worse—companies that have outright stolen someone else’s AI.

This investigative report comes with three key services:

1️⃣ I’m going to assist OAGs, OIGs, and other regulators across the country by exposing AI companies that make false claims about the accuracy of their health AI products.

2️⃣ I’ll break down, statistically, why you should never trust claims of 100% or even 90% accuracy from an AI model—at least not without serious skepticism.

3️⃣ I’ll arm you with 6 key questions to ask if someone tries to sell you on “90% AI accuracy.”

As always, I’m open to strong feedback—something along the lines of: “You jerk, you don’t know what you’re talking about. Here’s the empirical proof. Here’s the statistically validated, peer-reviewed research. Here’s the data.” If I’m wrong, I’m man enough to apologize. It’s happened before. I called out a company, and the CEO reached out—not to call me a jerk, though maybe he wanted to—but to provide research that defended their claims. While I wasn’t 100% convinced, I still apologized. We’re all human, and part of the same community, so respect and courtesy are non-negotiable in my book.

That said, if history is any guide, I tend to be spot-on in my analysis. I dedicate a lot of time to investigation, extraction, and fact-checking.

But before we dive into this critical topic, a few housekeeping items...

If you’re on Twitter/X and are free tonight, Wed, Oct 2, 2024, at 8 pm EST, I'll be debating Sanat Dixit MD, MBA, FACS on "Will healthcare AI replace physicians?" You can join here: https://x.com/i/spaces/1gqxvNBbmmexB or via my Twitter/X account @AIHealthUncut.

I have to say, this past week after my article dropped was one of the wildest yet. 'Selling The New o1 ‘Strawberry’ to Healthcare Is a Crime, No Matter What OpenAI Says 🍓' went viral—reprints, restacks, reposts, and fiery debates everywhere. Let me just be clear: I still stand by my core argument that OpenAI misrepresented the o1 model’s abilities in medical diagnostics. That said, I respect all the opinions being thrown around, and there are fine people on both sides. (It sounded funnier in my head, but now I’m not so sure. 😊)

I also want to give a shoutout to those who cited my article and sparked some great discussions, regardless of whether they agreed with me or not.

🔹 I’m honored to have been invited to co-author this article with the esteemed expert and star author, Devansh, for his renowned Substack publication, Artificial Intelligence Made Simple. This publication covers everything you need to know about the hottest AI topics and the complexities of machine learning models.

🔹 The article also went viral, prompting Devansh to write a follow-up where he dove even deeper into OpenAI’s apparent sloppiness in applying their newest o1 Strawberry model to medical diagnostics.

🔹 In a brilliant article that cites both my work and Devansh’s, James Wang argues that “connectionist reasoning” approaches like LLMs may not be the right path for fields like law and medicine, where being “approximately right” just doesn’t cut it.

🔹 Another fascinating perspective comes from Jurgen Gravestein in his piece “The AI Bubble.” He discusses AI expectations versus AI reality, with the OpenAI o1 Strawberry blunder being a prime example, and the post-bubble realism emerging in its wake.

🔹 Dr. Terence Tan, an AWS “healthcare wrangler,” wrote a thoughtful LinkedIn post summarizing my joint article with Devansh. His TL;DR sparked a lot of provocative discussions.

🔹 Yudara Kularathne MD, FAMS(EM), an ER physician and CEO of HeHealth, highlighted the significance of these discussions in a provocative LinkedIn post about AI’s role in healthcare.

🔹 David Talby, CTO of John Snow Labs, a health-focused AI company, summarized my article in a LinkedIn post that drew a lot of attention.

🔹 Pramodith, an AI engineer, wrote a compelling LinkedIn post about my article, emphasizing that, just like with humans, no matter how convincing or seemingly rational the AI’s chain of thought (CoT) may be, it can still be wrong.

Thank you, Devansh, James Wang, Dr. Terence Tan, Yudara Kularathne MD, FAMS(EM), David Talby, Pramodith, and many others for supporting and contributing to this crucial area of AI development in medicine.

Alright, enough with the pleasantries. Let’s cut to the chase…

This article continues my critique of OpenAI’s marketing push to convince everyone that AI models are ready for healthcare. They’re not.

Big tech players like OpenAI, IBM, and especially Epic, should be setting the standard for AI in healthcare. Instead, they’ve done the opposite.

It’s a huge problem for the healthcare industry. If these companies aren’t held accountable for their sloppy practices and lack of ethics in healthcare AI development, every other developer will assume it’s fine to cut corners and throw half-baked AI models into the market.

This behavior from the tech industry has tarnished AI’s reputation in the medical community.

AI was supposed to revolutionize medicine. Now, thanks to the reckless actions of the IT industry, it’s on pause.

I've investigated 15 companies and organizations that claim to contribute to healthcare through AI. But, whether by intent or incompetence, they're misrepresenting what they’re developing and selling, putting patients and doctors at risk. Many of the facts I uncovered, which I’m sharing exclusively in this article, can’t be found in corporate media because—surprise, surprise—you actually have to do the work. The so-called health tech journalists, with their deadlines, lattes, cushy 9-to-5 schedules, and “revolving doors” into the magical world of venture capital, lack the stamina or guts to do their job.

So why am I doing it for them? Honestly, I have no idea. But here we are.

Here’s the outline of this two-part article:

1. Pieces Technologies Faced Landmark Lawsuit Over False Claims of Health AI Model Accuracy

2. OpenAI: Shoving AI into Healthcare Before It’s Ready

3. IBM Watson Health: The ‘Lean Startup’ That Went Lean Right Off a Cliff

4. Epic’s AI: Cheating on Accuracy, Cashing In on Hype

5. Babylon Health: The Madoff of Digital Health That Slipped Through the Law’s Clumsy Fingers

6. Suki: (Allegedly) Stole AI From Google, Raised $165 Million—What Could Go Wrong?

7. South Korean Study Claims 100% Accuracy Diagnosing Autism with AI—Overfitting Called, It Wants Its Hype Back

8. Google: “90% Accuracy” My A** – Distribution Shift Is a Bitch, Ain’t It?

9. Infermedica’s Accuracy Claim: When Smoke and Mirrors Replace Peer Review

10. Sniffle’s Aignosis: When Your Diagnosis is Just a ‘Sniff’ Away From Certainty!

11. Isabel: The Accuracy Claims Nobody Else Wanted to Confirm

12. MayaMD: Outperforming ER Docs... According to MayaMD’s Own Study

13. Why Visit a Doctor When Klick Labs Can Misdiagnose You With a Sentence?

14. Did ‘Medical Chat’ Just Cheat Its Way Through the USMLE? Asking for a Friend.

15. OpenAI’s GPT-4o Scores High on USMLE. But Was It Playing With a Stacked Deck?

16. The $9 Per Hour AI Nurse: How Hippocratic AI’s ‘Healthcare Revolution’ Missed the Memo on Quality

17. The ‘Whisper’ That Shouted Nonsense: How OpenAI and Nabla’s AI Went Rogue in Healthcare

18. The Boldest Health AI Lies: How GPT-4, PaLM 2, Claude, and Llama are Selling Misinformation

19. Here’s What You Should Do When Someone Tries to Sell You “99% AI Accuracy”

20. My Take

Now, let’s dive in. (As I said, I still don’t know what the hell health tech journalists are doing. Why am I spending my weekends on these investigations while they regurgitate whatever health tech companies put in their press releases, trying to stay on their good side? It’s a question I keep asking myself, and I’m guessing it’ll go unanswered forever.)