Model-based reconstruction is a powerful framework for solving a variety of inverse problems in imaging. In recent years, enormous progress has been made in the problem of denoising, a special case of an inverse problem where the forward model is an identity operator. Similarly, great progress has been made in improving model-based inversion when the forward model corresponds to complex physical measurements in applications such as X-ray CT, electron-microscopy, MRI, and ultrasound, to name just a few. However, combining state-of-the-art denoising algorithms (i.e., prior models) with state-of-the-art inversion methods (i.e., forward models) has been a challenge for many reasons. In this paper, we propose a flexible framework that allows state-of-the-art forward models of imaging systems to be matched with state-of-the-art priors or denoising models. This framework, which we term as Plug-and-Play priors, has the advantage that it dramatically simplifies software integration, and moreover, it allows state-of-the-art denoising methods that have no known formulation as an optimization problem to be used. We demonstrate with some simple examples how Plug-and-Play priors can be used to mix and match a wide variety of existing denoising models with a tomographic forward model, thus greatly expanding the range of possible problem solutions.

Many material and biological samples in scientific imaging are characterized by nonlocal repeating structures. These are studied using scanning electron microscopy and electron tomography. Sparse sampling of individual pixels in a two-dimensional image acquisition geometry, or sparse sampling of projection images with large tilt increments in a tomography experiment, can enable high speed data acquisition and minimize sample damage caused by the electron beam. In this paper, we present an algorithm for electron tomographic reconstruction and sparse image interpolation that exploits the nonlocal redundancy in images. We adapt a framework, termed plug-and-play priors, to solve these imaging problems in a regularized inversion setting. The power of the plug-and-play approach is that it allows a wide array of modern denoising algorithms to be used as a “prior model” for tomography and image interpolation. We also present sufficient mathematical conditions that ensure convergence of the plug-and-play approach, and we use these insights to design a new nonlocal means denoising algorithm. Finally, we demonstrate that the algorithm produces higher quality reconstructions on both simulated and real electron microscope data, along with improved convergence properties compared to other methods.

Synchrotron X-ray computed tomography (SXCT) is increasingly being used for 3-D imaging of material samples at micron and finer scales. The success of these techniques has increased interest in 4-D reconstruction methods that can image a sample in both space and time. However, the temporal resolution of widely used 4-D reconstruction methods is severely limited by the need to acquire a very large number of views for each reconstructed 3-D volume. Consequently, the temporal resolution of current methods is insufficient to observe important physical phenomena. Furthermore, measurement nonidealities also tend to introduce ring and streak artifacts into the 4-D reconstructions. In this paper, we present a time-interlaced model-based iterative reconstruction (TIMBIR) method, which is a synergistic combination of two innovations. The first innovation, interlaced view sampling, is a novel method of data acquisition, which distributes the view angles more evenly in time. The second innovation is a 4-D model-based iterative reconstruction algorithm (MBIR), which can produce time-resolved volumetric reconstruction of the sample from the interlaced views. In addition to modeling both the sensor noise statistics and the 4-D object, the MBIR algorithm also reduces ring and streak artifacts by more accurately modeling the measurement nonidealities. We present reconstructions of both simulated and real X-ray synchrotron data, which indicate that TIMBIR can improve temporal resolution by an order of magnitude relative to existing approaches.

High angle annular dark field (HAADF)-scanning transmission electron microscope (STEM) data is increasingly being used in the physical sciences to research materials in 3D because it reduces the effects of Bragg diffraction seen in bright field TEM data. Typically, tomographic reconstructions are performed by directly applying either filtered back projection (FBP) or the simultaneous iterative reconstruction technique (SIRT) to the data. Since HAADF-STEM tomography is a limited angle tomography modality with low signal to noise ratio, these methods can result in significant artifacts in the reconstructed volume. In this paper, we develop a model based iterative reconstruction algorithm for HAADF-STEM tomography. We combine a model for image formation in HAADF-STEM tomography along with a prior model to formulate the tomographic reconstruction as a maximum a posteriori probability (MAP) estimation problem. Our formulation also accounts for certain missing measurements by treating them as nuisance parameters in the MAP estimation framework. We adapt the iterative coordinate descent algorithm to develop an efficient method to minimize the corresponding MAP cost function. Reconstructions of simulated as well as experimental data sets show results that are superior to FBP and SIRT reconstructions, significantly suppressing artifacts and enhancing contrast.

Synchrotron based X-ray tomography is widely used for three dimensional imaging of materials at the micron scale. Tomographic data collected from a synchrotron is often affected by non-idealities in the measurement system and sudden “blinding” of detector pixels during the acquisition. Typically, reconstructions are done using analytical reconstruction techniques combined with pre/post-processing steps to correct for the non-idealities, resulting in loss of detail while still producing noisy reconstructions with some artifacts. In this paper, we present a model-based iterative reconstruction (MBIR) algorithm for synchrotron X-ray tomography that can automatically handle the non-idealities as a part of the reconstruction. First, we develop a forward model that accounts for the non-idealities in the measurement system and for the occurrence of outliers in the measurement. Next, we combine the forward model with a prior model of the object to formulate the MBIR cost function and propose an algorithm to minimize the cost. Results on a real data set show that the MBIR reconstructions are superior to the analytical reconstructions effectively suppressing noise as well as other artifacts.