RIP Fine-Tuning in Medical AI: Data Is the King Again

OpenAI's HealthBench blew my mind: AI in medicine is becoming a Big, Beautiful, Glorified Prompt.

Welcome to AI Health Uncut, a brutally honest newsletter on AI, innovation, and the state of the healthcare market. If you’d like to sign up to receive issues over email, you can do so here.

Would you like to help students get full access to AI Health Uncut — and support my mission to improve healthcare through AI? Become a Founding Member.

Thinking of all my friends back in the U.S. 🇺🇸 dealing with the historic heat wave 🔥. Please stay safe 🙏. I’m lucky to still be in Lithuania 🇱🇹 for now, where the temperature has been a consistently cool 15–16°C ❄️🌿 throughout my stay so far.

Alright. Back to our regularly scheduled programming… 🎬

The autopsy is in, and the cause of death is clear. In a world where LLMs have flatlined and innovation has tapped out—especially in digital health—data has reclaimed the throne. Not just any data. High-quality, curated, real-world data. For years, customizing AI models with painstaking fine-tuning was the darling of the $5,000-a-ticket turtleneck VC conferences, hyped as the key to unlocking “AI doctors,” “AI nurses,” and “AI agents.” But as 2025 unfolds, a very-very different reality is emerging. In the blunt words of one commentator, fine-tuning turned out to be “sunk-cost theater with a GPU bill.” (Source: Srinivas Rao on Medium.) The latest evidence from both industry and research shows that real performance gains in medical AI now come from feeding powerful general models better data and prompts, not from fiddling with their internal neural net weights. This is an obituary for fine-tuning in healthcare AI. While data scientists seem to be completely distracted with SF apartment hunting and vibe coding, this is a call to refocus on what actually matters: the data.

The recent paper by OpenAI titled “HealthBench: Evaluating Large Language Models Towards Improved Human Health” opened my eyes to what we, as a healthcare industry, should actually expect from AI development.

My conclusion: Until AI developers stop sitting on their hands, doing useless vibe coding all day, and finally start doing something groundbreaking with math—yes, math, remember math? That’s what every line of code and every AI model is ultimately based on—nothing in healthcare AI is going to change. AI models are being completely commoditized. The LLM research frontier has stalled. This is excellent news for those who own the real gold: unique, high-quality, curated healthcare data.

We are NOT going to build meaningful AI agents using LLMs. I don’t care how loud Munjal Shah of Hippocratic AI or Hemant Taneja of General Catalyst shout from the stages of “Who Kisses the Most VC Asses” conferences. Wake up. We need actual innovation. Not more GPU farms. Not more AI tourists doing moodboard coding and selling LinkedIn hype. We need better math. We need better data.

Have you noticed how fast everyone forgot about Med-PaLM and Med-Gemini? These were painstakingly fine-tuned, medically-focused models, built for tens of millions of dollars. And yet they vanished from the conversation almost as soon as they were released—but not before Mayo Clinic and a few others signed contracts with Google. 😉

https://youtube.com/shorts/DVf3BFrhp_M

It’s only been a year. Just one year! And fine-tuning in medical AI is already ancient history.

Welcome to the Costco Era of Medical AI. Pick any model off the shelf. Get it in bulk. Just make sure you know how to prompt it—and more importantly, that you’ve got access to the one thing that actually matters: real, structured, high-quality medical data.

TL;DR:

1. OpenAI’s HealthBench: The Moment I Realized AI in Medicine Is Just Prompt Engineering

2. The Fine-Tuning Myth: Expensive Theater and Illusions of Progress

3. LLMs Are Commoditized: Off-the-Shelf Brains, Proprietary Data

4. Why Fine-Tuning Is Losing the War to Clean Data

5. Data Is King – Again (The Future in the Current “Lazy Vibe Coding” Reality Belongs to the Dataset, Not the Algorithm)

6. Conclusion: The Doctor’s New Prescription – Data-Driven AI