Fluorescence lifetime imaging (FLI) provides unique quantitative information in biomedical and molecular biology studies but relies on complex data-fitting techniques to derive the quantities of interest. Herein, we propose a fit-free approach in FLI image formation that is based on deep learning (DL) to quantify fluorescence decays simultaneously over a whole image and at fast speeds. We report on a deep neural network (DNN) architecture, named fluorescence lifetime imaging network (FLI-Net) that is designed and trained for different classes of experiments, including visible FLI and near-infrared (NIR) FLI microscopy (FLIM) and NIR gated macroscopy FLI (MFLI). FLI-Net outputs quantitatively the spatially resolved lifetime-based parameters that are typically employed in the field. We validate the utility of the FLI-Net framework by performing quantitative microscopic and preclinical lifetime-based studies across the visible and NIR spectra, as well as across the 2 main data acquisition technologies. These results demonstrate that FLI-Net is well suited to accurately quantify complex fluorescence lifetimes in cells and, in real time, in intact animals without any parameter settings. Hence, FLI-Net paves the way to reproducible and quantitative lifetime studies at unprecedented speeds, for improved dissemination and impact of FLI in many important biomedical applications ranging from fundamental discoveries in molecular and cellular biology to clinical translation.

Near-infrared (NIR) fluorescence lifetime imaging (FLI) provides a unique contrast mechanism to monitor biological parameters and molecular events in vivo. Single-photon avalanche diode (SPAD) cameras have been recently demonstrated in FLI microscopy (FLIM) applications, but their suitability for in vivo macroscopic FLI (MFLI) in deep tissues remains to be demonstrated. Herein, we report in vivo NIR MFLI measurement with SwissSPAD2, a large time-gated SPAD camera. We first benchmark its performance in well-controlled in vitro experiments, ranging from monitoring environmental effects on fluorescence lifetime, to quantifying Förster resonant energy transfer (FRET) between dyes. Next, we use it for in vivo studies of target-drug engagement in live and intact tumor xenografts using FRET. Information obtained with SwissSPAD2 was successfully compared to that obtained with a gated intensified charge-coupled device (ICCD) camera, using two different approaches. Our results demonstrate that SPAD cameras offer a powerful technology for in vivo preclinical applications in the NIR window.

The development of real-time, wide-field and quantitative diffuse optical imaging methods to visualize functional and structural biomarkers of living tissues is a pressing need for numerous clinical applications including image-guided surgery. In this context, Spatial Frequency Domain Imaging (SFDI) is an attractive method allowing for the fast estimation of optical properties using the Single Snapshot of Optical Properties (SSOP) approach. Herein, we present a novel implementation of SSOP based on a combination of deep learning network at the filtering stage and Graphics Processing Units (GPU) capable of simultaneous high visual quality image reconstruction, surface profile correction and accurate optical property (OP) extraction in real-time across large fields of view. In the most optimal implementation, the presented methodology demonstrates megapixel profile-corrected OP imaging with results comparable to that of profile-corrected SFDI, with a processing time of 18 ms and errors relative to SFDI method less than 10% in both profilometry and profile-corrected OPs. This novel processing framework lays the foundation for real-time multispectral quantitative diffuse optical imaging for surgical guidance and healthcare applications. All code and data used for this work is publicly available at www.healthphotonics.org under the resources tab.

Hyperspectral fluorescence lifetime imaging allows for the simultaneous acquisition of spectrally resolved temporal fluorescence emission decays. In turn, the acquired rich multidimensional data set enables simultaneous imaging of multiple fluorescent species for a comprehensive molecular assessment of biotissues. However, to enable quantitative imaging, inherent spectral overlap between the considered fluorescent probes and potential bleed-through must be considered. Such a task is performed via either spectral or lifetime unmixing, typically independently. Herein, we present “UNMIX-ME” (unmix multiple emissions), a deep learning-based fluorescence unmixing routine, capable of quantitative fluorophore unmixing by simultaneously using both spectral and temporal signatures. UNMIX-ME was trained and validated using an in silico framework replicating the data acquisition process of a compressive hyperspectral fluorescent lifetime imaging platform (HMFLI). It was benchmarked against a conventional LSQ method for tri and quadri-exponential simulated samples. Last, UNMIX-ME’s potential was assessed for NIR FRET in vitro and in vivo preclinical applications.

Significance: Biomedical optics system design, image formation, and image analysis have primarily been guided by classical physical modeling and signal processing methodologies. Recently, however, deep learning (DL) has become a major paradigm in computational modeling and has demonstrated utility in numerous scientific domains and various forms of data analysis. Aim: We aim to comprehensively review the use of DL applied to macroscopic diffuse optical imaging (DOI). Approach: First, we provide a layman introduction to DL. Then, the review summarizes current DL work in some of the most active areas of this field, including optical properties retrieval, fluorescence lifetime imaging, and diffuse optical tomography. Results: The advantages of using DL for DOI versus conventional inverse solvers cited in the literature reviewed herein are numerous. These include, among others, a decrease in analysis time (often by many orders of magnitude), increased quantitative reconstruction quality, robustness to noise, and the unique capability to learn complex end-to-end relationships. Conclusions: The heavily validated capability of DL’s use across a wide range of complex inverse solving methodologies has enormous potential to bring novel DOI modalities, otherwise deemed impractical for clinical translation, to the patient’s bedside.

We report on a macroscopic fluorescence lifetime imaging (MFLI) topography computational framework based around machine learning with the main goal of retrieving the depth of fluorescent inclusions deeply seated in bio-tissues. This approach leverages the depth-resolved information inherent to time-resolved fluorescence data sets coupled with the retrieval of in situ optical properties as obtained via spatial frequency domain imaging (SFDI). Specifically, a Siamese network architecture is proposed with optical properties (OPs) and time-resolved fluorescence decays as input followed by simultaneous retrieval of lifetime maps and depth profiles. We validate our approach using comprehensive in silico data sets as well as with a phantom experiment. Overall, our results demonstrate that our approach can retrieve the depth of fluorescence inclusions, especially when coupled with optical properties estimation, with high accuracy. We expect the presented computational approach to find great utility in applications such as optical-guided surgery.