Data Problems in Diagnosis Land

The data problem

LLMs learn from text — medical literature, forums, patient descriptions — not from actually examining patients. A real diagnosis relies heavily on things that can’t be typed: how someone looks, sounds, moves, their skin color, whether they wince when you press somewhere. That’s just absent.

Symptoms are wildly non-specific

“Fatigue, headache, and nausea” maps to hundreds of conditions — from a hangover to a brain tumor. A doctor narrows this down through physical exam, lab work, imaging, and clinical intuition built from seeing thousands of patients. An LLM has none of those tools, so it’s guessing from incomplete information.

People describe symptoms poorly (through no fault of their own)

Patients don’t know what’s medically relevant. They might not mention that symptoms are worse at night, or that their father had the same thing, or that they started a new medication. A skilled clinician knows what follow-up questions to ask and how to weight the answers. An LLM in a chatbot doesn’t reliably do this.

Hallucination and overconfidence

LLMs can generate plausible-sounding but wrong medical conclusions confidently. In a low-stakes domain, that’s annoying. In medicine, it can cause real harm — either by falsely reassuring someone with a serious condition, or by alarming someone with something benign.

No continuity or context

A doctor who knows you — your history, your medications, your lifestyle — has a huge advantage. A chatbot typically starts cold every time.

Base rate blindness

Good diagnosis requires knowing how common things are in a specific population. A 25-year-old with chest pain is statistically very different from a 60-year-old with chest pain, even with identical symptom descriptions. LLMs can struggle to appropriately weight these population-level priors in context.

The most honest summary: diagnosis is fundamentally an embodied, relational, probabilistic process with real-world feedback loops. LLMs are pattern-matching over text. Those two things are quite far apart, even when the LLM sounds very authoritative.​​​​​​​​​​​​​​​​