29 June 2020 in Nature Biotechnology
The contrast and resolution of images obtained with optical microscopes can be improved by deconvolution and computational fusion of multiple views of the same sample, but these methods are computationally expensive for large datasets. Here we describe theoretical and practical advances in algorithm and software design that result in image processing times that are tenfold to several thousand fold faster than with previous methods. First, we show that an ‘unmatched back projector’ accelerates deconvolution relative to the classic Richardson–Lucy algorithm by at least tenfold. Second, three-dimensional image-based registration with a graphics processing unit enhances processing speed 10- to 100-fold over CPU processing. Third, deep learning can provide further acceleration, particularly for deconvolution with spatially varying point spread functions. We illustrate our methods from the subcellular to millimeter spatial scale on diverse samples, including single cells, embryos and cleared tissue. Finally, we show performance enhancement on recently developed microscopes that have improved spatial resolution, including dual-view cleared-tissue light-sheet microscopes and reflective lattice light-sheet microscopes.
26 November 2021 in Nature
26 January 2023 in Nature Biotechnology
The axial resolution of three-dimensional structured illumination microscopy (3D SIM) is limited to ∼300 nm. Here we present two distinct, complementary methods to improve axial resolution in 3D SIM with minimal or no modification to the optical system. We show that placing a mirror directly opposite the sample enables four-beam interference with higher spatial frequency content than 3D SIM illumination, offering near-isotropic imaging with ∼120-nm lateral and 160-nm axial resolution. We also developed a deep learning method achieving ∼120-nm isotropic resolution. This method can be combined with denoising to facilitate volumetric imaging spanning dozens of timepoints. We demonstrate the potential of these advances by imaging a variety of cellular samples, delineating the nanoscale distribution of vimentin and microtubule filaments, observing the relative positions of caveolar coat proteins and lysosomal markers and visualizing cytoskeletal dynamics within T cells in the early stages of immune synapse formation.
2 January 2025 in Nature Communications
Optical aberrations hinder fluorescence microscopy of thick samples, reducing image signal, contrast, and resolution. Here we introduce a deep learning-based strategy for aberration compensation, improving image quality without slowing image acquisition, applying additional dose, or introducing more optics. Our method (i) introduces synthetic aberrations to images acquired on the shallow side of image stacks, making them resemble those acquired deeper into the volume and (ii) trains neural networks to reverse the effect of these aberrations. We use simulations and experiments to show that applying the trained ‘de-aberration’ networks outperforms alternative methods, providing restoration on par with adaptive optics techniques; and subsequently apply the networks to diverse datasets captured with confocal, light-sheet, multi-photon, and super-resolution microscopy. In all cases, the improved quality of the restored data facilitates qualitative image inspection and improves downstream image quantitation, including orientational analysis of blood vessels in mouse tissue and improved membrane and nuclear segmentation in C. elegans embryos.
15 July 2019 in Journal of the Optical Society of America A
We introduce the basic elements of a spatio-angular theory of fluorescence microscopy, providing a unified framework for analyzing systems that image single fluorescent dipoles and ensembles of overlapping dipoles that label biological molecules. We model an aplanatic microscope imaging an ensemble of fluorescent dipoles as a linear Hilbert-space operator, and we show that the operator takes a particularly convenient form when expressed in a basis of complex exponentials and spherical harmonics—a form we call the dipole spatio-angular transfer function. We discuss the implications of our analysis for all quantitative fluorescence microscopy studies and lay out a path toward a complete theory.