Diffusion beat GANs in medical imaging | Here's the actual mechanism
5 min readmachine-learning
Diffusion models didn't replace GANs for synthetic medical imaging because they're newer. They replaced them because of what mode collapse costs you when the whole point is covering the tail of the distribution.
A follow-up worth being precise about
In the post on GAN evaluation metrics I went through what FID and Inception Score actually measure, and why both inherit assumptions from ImageNet that break on biomedical images. That post didn’t touch the question of architecture — GAN vs something else. This one does, because the “something else” has mostly settled: diffusion models have displaced GANs as the default for synthetic medical imaging, and it’s worth being precise about why, because the lazy version of the answer (“diffusion is newer, newer wins”) is wrong in a way that matters for whether you should actually switch.
What the foundational comparison actually says
Dhariwal and Nichol’s 2021 paper, the one usually cited for “diffusion models beat GANs,” is not making a claim about sample sharpness alone. The mechanism they point to is specific: GANs are trained via an adversarial game between generator and discriminator, which is a two-player optimization that’s famously unstable to balance, and prone to mode collapse — the generator finding a narrow slice of the data distribution that reliably fools the discriminator, and settling there rather than covering the full distribution it’s supposed to be modeling. Diffusion models replace that adversarial game with a single, stable training objective — predicting noise at each denoising step — with no second network to equilibrate against. There’s no Nash equilibrium to find, because there’s no adversary.
That training-stability gap is the actual mechanism. It’s not that diffusion models draw sharper individual images — GANs can and often do match or beat them there. It’s that they reliably cover more of the underlying distribution, because nothing in their training process rewards collapsing onto a convenient subset.
Why that specific gap matters more in medical imaging than anywhere else
Picture what synthetic data augmentation is actually for in a biomedical context: covering the cases you don’t have enough real examples of — rare presentations, uncommon imaging artifacts, unusual anatomy, the long tail a training set is thinnest on. That is precisely the generation task mode collapse quietly ruins. A GAN that’s mode-collapsed produces visually convincing images — that’s the trap. You can look at a grid of its output and see nothing wrong, because every image in the grid is individually plausible. What’s missing is everything the generator decided not to bother with, and a grid of samples will never show you an absence.
This is exactly the blind spot the GAN evaluation post was about: FID and IS each compress fidelity and diversity into one number, and a generator that’s memorized a narrow subset can score fine on fidelity while failing coverage badly. Mode collapse in a medical-imaging GAN isn’t a cosmetic problem you’d notice by looking. It’s a coverage problem you only catch if you specifically measure for it — and the whole value of the synthetic data was the coverage.
Medical-imaging-specific follow-up work confirms the general finding transfers to this domain directly: diffusion probabilistic models outperform GANs on medical images specifically, and a 2024 RSNA Radiology review on synthetic data generation for medical imaging catalogs diffusion models being used to generate chest radiography, cardiac MRI, pelvic CT, and abdominal CT for classifier training — the use case is exactly the dataset-enrichment one where coverage is the point.
A caveat from my own dissertation work
During my PhD research on synthesizing biomedical images, we ran Stable Diffusion directly on cytological images and it performed worse than the GAN baselines being compared at the time — not on mode coverage, but on basic plausibility. The color palette looked right; cell shapes didn’t match real cytology, because the model injects a “creative” prior that’s actively unwanted here. That’s an important caveat to state plainly: “diffusion beats GANs” in the Dhariwal & Nichol sense is about models trained directly on the target image domain, architecture vs. architecture. A general-purpose text-to-image model like Stable Diffusion is a different comparison entirely — it’s bringing a strong natural-image prior that actively fights a domain with no natural-image structure to borrow from. Don’t read this post as “point Stable Diffusion at your medical images” — that’s not what the evidence says, including my own.
The actual trade-off, honestly stated
None of this makes GANs obsolete. Diffusion’s cost is real: generating a sample means running many sequential denoising steps, against a GAN’s single forward pass. For an interactive tool — something a clinician or researcher is using live, waiting on a result — that latency difference is the whole ballgame, and a GAN is often still the right call. For offline dataset augmentation, where you generate a batch once and use it to train a classifier, the per-sample speed difference barely registers, and the coverage advantage is what you’re actually buying.
So the decision isn’t “diffusion is better, use it.” It’s: what’s the binding constraint — tail coverage, or inference latency? Offline augmentation for a training set: diffusion, because mode collapse is the exact failure you can’t afford. Interactive generation where someone’s waiting: weigh the latency cost against how much coverage you actually need for that use case.
The provenance question this creates
If synthetic data generated this way enters a training pipeline that eventually produces a model pointed at a patient, the clinical MLOps post applies directly: “which generator, which checkpoint, what coverage evidence” becomes an audit question, not just a modeling choice. A FID number — computed in whatever feature space, per the evaluation post — tells you the generator is a reasonable augmentation source. It doesn’t tell you the resulting classifier is safe to deploy. Those stay separate gates, architecture choice included.
Sources: Dhariwal & Nichol, “Diffusion Models Beat GANs on Image Synthesis” (arXiv 2105.05233); “Diffusion Probabilistic Models Beat GANs on Medical Images” (arXiv 2212.07501); RSNA Radiology, “Generating Synthetic Data for Medical Imaging”; PMC systematic review on generative models for medical image creation and translation; the existing GAN-evaluation-metrics post for the metrics vocabulary.