A year ago we migrated every indication head in the platform off bespoke CNN backbones and onto a shared foundation-model encoder pretrained on roughly 180 million tiles drawn from public and partner-contributed cohorts. The headline improvement on internal benchmarks was real. The migration itself was harder than we expected, and the engineering cost is worth writing about in detail.
What transferred cleanly
Representation quality transferred. Tasks that previously required 50,000 annotated tiles per indication now reach equivalent performance on roughly a tenth of that. More importantly, the representations are stable across scanners in a way that end-to-end CNN backbones never quite achieved for us.
What did not
Calibration did not transfer. The foundation backbone produces well-separated features but its downstream classifiers tend to be overconfident on out-of-distribution tiles. We addressed this with a per-indication temperature calibration step run against a held-out multi-site calibration cohort, which is now part of our release pipeline.
Memory footprint also did not transfer quietly. The foundation encoder is roughly four times the size of the CNN backbones it replaced. On our standard inference node that meant rewriting the batching layer to use feature caching and scheduling.
Where we landed
On balance the migration paid for itself within six months, but not through raw accuracy. The real return was development velocity: shipping a new indication head is now a two-to-four week exercise for a small team, rather than a three-to-six month one. That is the number that matters to our partners.
