28 August 2020 in Neurophotonics
Cellular resolution voltage imaging enables direct observation of neuronal computation. Indeed, membrane potential imaging experiments have spatiotemporally resolved both active and passive action, and synaptic potential generation throughout dendritic and axonal arbors.1 – 14 Resolution of these small voltage signals at high speeds requires high photon fluxes, making wide-field single-photon (1P) imaging by far the most common voltage imaging modality. Imaging neuronal processes with this technique require the imaged membranes to lie approximately flat in the microscope’s focal plane. As these experiments are typically performed in slices, the requirement for flat, healthy, and superficial cells represents a significant barrier to entry for experimenters. Even in the best-prepared slices, anatomy dictates that only a few cells will be oriented parallel to the surface, reducing experimental throughput, and only certain cell types feature morphology that can be well-sampled by a single plane. Iterative three-dimensional (3-D) imaging in wide field via physical refocusing is often untenable due to rapid sample bleaching. Multiple approaches to improve wide-field imaging’s 3-D performance have been developed. Anselmi et al.15 applied remote focusing to axially shift and tilt the wide-field focal plane as required by the sample, enabling calcium imaging along tilted dendrites. This adaptation, however, costs half of the fluorescence emission and is limited to a single tilted plane at a time. Point spread function (PSF) engineering via cubic phase masks16 or spherical aberration17 also enables parallelized volumetric sample imaging when combined with light-sheet excitation; however, to our knowledge, these approaches have not successfully been implemented to image membrane voltage.
29 May 2020 in IEEE Transactions on Computational Imaging
Light-field microscopy (LFM) is a type of all-optical imaging system that is able to capture 4D geometric information of light rays and can reconstruct a 3D model from a single snapshot. In this paper, we propose a new 3D localization approach to effectively detect 3D positions of neuronal cells from a single light-field image with high accuracy and outstanding robustness to light scattering. This is achieved by constructing a depth-aware dictionary and by combining it with convolutional sparse coding. Specifically, our approach includes 3 key parts: light-field calibration, depth-aware dictionary construction, and localization based on convolutional sparse coding (CSC). In the first part, an observed raw light-field image is calibrated and then decoded into a two-plane parameterized 4D format which leads to the epi-polar plane image (EPI). The second part involves simulating a set of light-fields using a wave-optics forward model for a ball-shaped volume that is located at different depths. Then, a depth-aware dictionary is constructed where each element is a synthetic EPI associated to a specific depth. Finally, by taking full advantage of the sparsity prior and shift-invariance property of EPI, 3D localization is achieved via convolutional sparse coding on an observed EPI with respect to the depth-aware EPI dictionary. We evaluate our approach on both non-scattering specimen (fluorescent beads suspended in agarose gel) and scattering media (brain tissues of genetically encoded mice). Extensive experiments demonstrate that our approach can reliably detect the 3D positions of granular targets with small Root Mean Square Error (RMSE), high robustness to optical aberration and light scattering in mammalian brain tissues.
1 June 2023 in IEEE Transactions on Computational Imaging
Light Field Microscopy (LFM) is an imaging technique that offers the opportunity to study fast dynamics in biological systems due to its 3D imaging speed and is particularly attractive for functional neuroimaging. Traditional model-based approaches employed in microscopy for reconstructing 3D images from light-field data are affected by reconstruction artifacts and are computationally demanding. This work introduces a deep neural network for LFM to image neuronal activity under adverse conditions: limited training data, background noise, and scattering mammalian brain tissue. The architecture of the network is obtained by unfolding the ISTA algorithm and is based on the observation that neurons in the tissue are sparse. Our approach is also based on a novel modelling of the imaging system that uses a linear convolutional neural network to fit the physics of the acquisition process. We train the network in a semi-supervised manner based on an adversarial training framework. The small labelled dataset required for training is acquired from a single sample via two-photon microscopy, a point-scanning 3D imaging technique that achieves high spatial resolution and deep tissue penetration but at a lower speed than LFM. We introduce physics knowledge of the system in the design of the network architecture and during training to complete our semi-supervised approach. We experimentally show that in the proposed scenario, our method performs better than typical deep learning and model-based reconstruction strategies for imaging neuronal activity in mammalian brain tissue via LFM, considering reconstruction quality, generalization to functional imaging, and reconstruction speed.
18 March 2022 in IEEE Transactions on Computational Imaging
Light Field Microscopy (LFM) is an imaging technique that captures 3D spatial information with a single 2D image. LFM is attractive because of its relatively simple implementation and fast volume acquisition rate. Capturing volume time series at a camera frame rate can enable the study of the behaviour of many biological systems. For instance, it could provide insights into the communication dynamics of living 3D neural networks. However, conventional 3D reconstruction algorithms for LFM typically suffer from high computational cost, low lateral resolution, and reconstruction artifacts. In this work, we study the origin of these issues and propose novel techniques to improve the performance of the reconstruction process. First, we propose a discretization approach that uses shift-invariant subspaces to generalize the typical discretization framework used in LFM. Then, we study the shift-invariant-subspace assumption as a prior for volume reconstruction under ideal conditions. Furthermore, we present a method to reduce the computational time of the forward model by using singular value decomposition (SVD). Finally, we propose to use iterative approaches that incorporate additional priors to perform artifact-free 3D reconstruction from real light field images. We experimentally show that our approach performs better than Richardson-Lucy-based strategies in computational time, image quality, and artifact reduction.
24 April 2024 in IEEE Transactions on Image Processing
In modern neuroscience, observing the dynamics of large populations of neurons is a critical step of understanding how networks of neurons process information. Light-field microscopy (LFM) has emerged as a type of scanless, high-speed, three-dimensional (3D) imaging tool, particularly attractive for this purpose. Imaging neuronal activity using LFM calls for the development of novel computational approaches that fully exploit domain knowledge embedded in physics and optics models, as well as enabling high interpretability and transparency. To this end, we propose a model-based explainable deep learning approach for LFM. Different from purely data-driven methods, the proposed approach integrates wave-optics theory, sparse representation and non-linear optimization with the artificial neural network. In particular, the architecture of the proposed neural network is designed following precise signal and optimization models. Moreover, the network’s parameters are learned from a training dataset using a novel training strategy that integrates layer-wise training with tailored knowledge distillation. Such design allows the network to take advantage of domain knowledge and learned new features. It combines the benefit of both model-based and learning-based methods, thereby contributing to superior interpretability, transparency and performance. By evaluating on both structural and functional LFM data obtained from scattering mammalian brain tissues, we demonstrate the capabilities of the proposed approach to achieve fast, robust 3D localization of neuron sources and accurate neural activity identification.
18 May 2025 in Medical Image Analysis
Super-resolution ultrasound (SRUS) imaging through localisation and tracking of microbubble (MB), also known as ultrasound localisation microscopy (ULM), allows non-invasive imaging of microvasculature in vivo beyond the diffraction limit. The number of MBs localised from the acquired contrast-enhanced ultrasound (CEUS) images and the localisation accuracy precision directly influence the quality of the resulting super-resolution microvasculature images. However, non-negligible noise present in the CEUS images can make localising MBs challenging. To enhance the MB localisation performance, we propose a Multi-Frame Deconvolution (MF-Decon) framework that can exploit the spatiotemporal consistency inherent in the CEUS data, with new spatial and temporal regularisers designed based on total variation (TV) and regularisation by denoising (RED). Based on the MF-Decon framework, we introduce two novel methods: MF-Decon with spatial and temporal TVs (MF-Decon+3DTV) and MF-Decon with spatial RED and temporal TV (MF-Decon+RED+TV). Results from in silico simulations indicate that our methods outperform two widely used methods using deconvolution or normalised cross-correlation across all evaluation metrics, including precision, recall, F1 score, mean and standard localisation errors. In particular, our methods improve MB localisation precision by up to 39 % and recall by up to 12 %. Super-resolution microvasculature maps generated with our methods on a publicly available in vivo rat brain dataset show less noise, better contrast, higher resolution and more vessel structures.