Deep learning-based image processing is capable of creating highly appealing results. However, it is still widely considered as a "blackbox" transformation. In medical imaging, this lack of comprehensibility of the results is a sensitive issue. The integration of known operators into the deep learning environment has proven to be advantageous for the comprehensibility and reliability of the computations. Consequently, we propose the use of the locally linear guided filter in combination with a learned guidance map for general purpose medical image processing. The output images are only processed by the guided filter while the guidance map can be trained to be task-optimal in an end-to-end fashion. We investigate the performance based on two popular tasks: image super resolution and denoising. The evaluation is conducted based on pairs of multi-modal magnetic resonance imaging and cross-modal computed tomography and magnetic resonance imaging datasets. For both tasks, the proposed approach is on par with state-of-the-art approaches. Additionally, we can show that the input image's content is almost unchanged after the processing which is not the case for conventional deep learning approaches. On top, the proposed pipeline offers increased robustness against degraded input as well as adversarial attacks.
Hybrid X-ray and magnetic resonance (MR) imaging promises large potential in interventional medical imaging applications due to the broad variety of contrast of MRI combined with fast imaging of X-ray-based modalities. To fully utilize the potential of the vast amount of existing image enhancement techniques, the corresponding information from both modalities must be present in the same domain. For image-guided interventional procedures, X-ray fluoroscopy has proven to be the modality of choice. Synthesizing one modality from another in this case is an ill-posed problem due to ambiguous signal and overlapping structures in projective geometry. To take on these challenges, we present a learning-based solution to MR to X-ray projection-to-projection translation. We propose an image generator network that focuses on high representation capacity in higher resolution layers to allow for accurate synthesis of fine details in the projection images. Additionally, a weighting scheme in the loss computation that favors high-frequency structures is proposed to focus on the important details and contours in projection imaging. The proposed extensions prove valuable in generating X-ray projection images with natural appearance. Our approach achieves a deviation from the ground truth of only 6% and structural similarity measure of 0.913 ± 0.005. In particular the high frequency weighting assists in generating projection images with sharp appearance and reduces erroneously synthesized fine details.
13 April 2021 in IEEE Transactions on Medical Imaging
For the clinical assessment of cardiac vitality, time-continuous tomographic imaging of the heart is used. To further detect e.g., pathological tissue, multiple imaging contrasts enable a thorough diagnosis using magnetic resonance imaging (MRI). For this purpose, time-continous and multi-contrast imaging protocols were proposed. The acquired signals are binned using navigation approaches for a motion-resolved reconstruction. Mostly, external sensors such as electrocardiograms (ECG) are used for navigation, leading to additional workflow efforts. Recent sensor-free approaches are based on pipelines requiring prior knowledge, e.g., typical heart rates. We present a sensor-free, deep learning-based navigation that diminishes the need for manual feature engineering or the necessity of prior knowledge compared to previous works. A classifier is trained to estimate the R-wave timepoints in the scan directly from the imaging data. Our approach is evaluated on 3-D protocols for continuous cardiac MRI, acquired in-vivo and free-breathing with single or multiple imaging contrasts. We achieve an accuracy of > 98% on previously unseen subjects, and a well comparable image quality with the state-of-the-art ECG-based reconstruction. Our method enables an ECG-free workflow for continuous cardiac scans with simultaneous anatomic and functional imaging with multiple contrasts. It can be potentially integrated without adapting the sampling scheme to other continuous sequences by using the imaging data for navigation and reconstruction.
28 May 2020 in IEEE Transactions on Medical Imaging
X-ray imaging is a wide-spread real-time imaging technique. Magnetic Resonance Imaging (MRI) offers a multitude of contrasts that offer improved guidance to interventionalists. As such simultaneous real-time acquisition and overlay would be highly favorable for image-guided interventions, e.g., in stroke therapy. One major obstacle in this setting is the fundamentally different acquisition geometry. MRI k-space sampling is associated with parallel projection geometry, while the X-ray acquisition results in perspective distorted projections. The classical rebinning methods to overcome this limitation inherently suffers from a loss of resolution. To counter this problem, we present a novel rebinning algorithm for parallel to cone-beam conversion. We derive a rebinning formula that is then used to find an appropriate deep neural network architecture. Following the known operator learning paradigm, the novel algorithm is mapped to a neural network with differentiable projection operators enabling data-driven learning of the remaining unknown operators. The evaluation aims in two directions: First, we give a profound analysis of the different hypotheses to the unknown operator and investigate the influence of numerical training data. Second, we evaluate the performance of the proposed method against the classical rebinning approach. We demonstrate that the derived network achieves better results than the baseline method and that such operators can be trained with simulated data without losing their generality making them applicable to real data without the need for retraining or transfer learning.