A Diffusion Model Predicts 3D Shapes From 2D Microscopy Images

Diffusion models are a specific type of generative model, capable of synthesising new data from a learnt distribution. We introduce DISPR, a diffusion-based model for solving the inverse problem of three-dimensional (3D) cell shape prediction from two-dimensional (2D) single cell microscopy images. Using the 2D microscopy im- age as a prior, DISPR is conditioned to predict realistic 3D shape re- constructions. To showcase the applicability of DISPR as a data aug- mentation tool in a feature-based single cell classification task, we extract morphological features from the red blood cells grouped into six highly imbalanced classes. Adding features from the DISPR pre- dictions to the three minority classes improved the macro F1 score from F 1macro = 55.2 ± 4.6% to F 1macro = 72.2 ± 4.9%. We thus demonstrate that diffusion models can be successfully applied to in- verse biomedical problems, and that they learn to reconstruct 3D shapes with realistic morphological features from 2D microscopy images.