23 April 2019 in IEEE Transactions on Medical Imaging
Edges tend to be over-smoothed in total variation (TV) regularized under-sampled images. In this paper, symmetric residual convolutional neural network (SR-CNN), a deep learning based model, was proposed to enhance the sharpness of edges and detailed anatomical structures in under-sampled cone-beam computed tomography (CBCT). For training, CBCT images were reconstructed using TV-based method from limited projections simulated from the ground truth CT, and were fed into SR-CNN, which was trained to learn a restoring pattern from under-sampled images to the ground truth. For testing, under-sampled CBCT was reconstructed using TV regularization and was then augmented by SR-CNN. Performance of SR-CNN was evaluated using phantom and patient images of various disease sites acquired at different institutions both qualitatively and quantitatively using structure similarity (SSIM) and peak signal-to-noise ratio (PSNR). SR-CNN substantially enhanced image details in the TV-based CBCT across all experiments. In the patient study using real projections, SR-CNN augmented CBCT images reconstructed from as low as 120 half-fan projections to image quality comparable to the reference fully-sampled FDK reconstruction using 900 projections. In the tumor localization study, improvements in the tumor localization accuracy were made by the SR-CNN augmented images compared with the conventional FDK and TV-based images. SR-CNN demonstrated robustness against noise levels and projection number reductions and generalization for various disease sites and datasets from different institutions. Overall, the SR-CNN-based image augmentation technique was efficient and effective in considerably enhancing edges and anatomical structures in under-sampled 3D/4D-CBCT, which can be very valuable for image-guided radiotherapy.
7 December 2021 in IEEE Transactions on Radiation and Plasma Medical Sciences
4D-CBCT is a powerful tool to provide respiration-resolved images for the moving target localization. However, projections in each respiratory phase are intrinsically under-sampled under the clinical scanning time and imaging dose constraints. Images reconstructed by compressed sensing (CS)-based methods suffer from blurred edges. Introducing the average-4D-image constraint to the CS-based reconstruction, such as prior-image-constrained CS (PICCS), can improve the edge sharpness of the stable structures. However, PICCS can lead to motion artifacts in the moving regions. In this study, we proposed a dual-encoder convolutional neural network (DeCNN) to realize the average-image-constrained 4D-CBCT reconstruction. The proposed DeCNN has two parallel encoders to extract features from both the under-sampled target phase images and the average images. The features are then concatenated and fed into the decoder for the high-quality target phase image reconstruction. The reconstructed 4D-CBCT using of the proposed DeCNN from the real lung cancer patient data showed: 1) qualitatively, clear and accurate edges for both stable and moving structures; 2) quantitatively, low-intensity errors, high peak signal-to-noise ratio, and high structural similarity compared to the ground truth images; and 3) superior quality to those reconstructed by several other state-of-the-art methods, including the back-projection, CS total variation, PICCS, and the single-encoder convolutional neural network. Overall, the proposed DeCNN is effective in exploiting the average-image constraint to improve the 4D-CBCT image quality.
22 September 2024 in Precision Radiation Oncology
Background: Nanosecond pulsed electric fields (nsPEF)‐based electroporation is a new therapy modality potentially synergized with radiation therapy to improve treatment outcomes. To verify its treatment accuracy intraoperatively, electroacoustic tomography (EAT) has been developed to monitor in‐vivo electric energy deposition by detecting ultrasound signals generated by nsPEFs in real‐time. However, utility of EAT is limited by image distortions due to the limited‐angle view of ultrasound transducers.Methods: This study proposed a supervised learning‐based workflow to address the ill‐conditioning in EAT reconstruction. Electroacoustic signals were detected by a linear array and initially reconstructed into EAT images, which were then fed into a deep learning model for distortion correction. In this study, 56 distinct electroacoustic data sets from nsPEFs of different intensities and geometries were collected experimentally, avoiding simulation‐to‐real‐world variations. Forty‐six data were used for model training and 10 for testing. The model was trained using supervised learning, enabled by a custom rotating platform to acquire paired full‐view and single‐view signals for the same electric field.Results: The proposed method considerably improved the image quality of linear array‐based EAT, generating pressure maps with accurate and clear structures. Quantitatively, the enhanced single‐view images achieved a low‐intensity error (RMSE: 0.018), high signal‐to‐noise ratio (PSNR: 35.15), and high structural similarity (SSIM: 0.942) compared to the reference full‐view images.Conclusions: This study represented a pioneering stride in achieving high‐quality EAT using a single linear array in an experimental environment, which improves EAT's utility in real‐time monitoring for nsPEF‐based electroporation therapy.
21 January 2026 in Physics in Medicine & Biology
Objective. Unsupervised deep learning has shown great promise in deformable image registration (DIR). These methods update model weights to optimize image similarity without requiring ground truth deformation vector fields (DVFs). However, they inherently face the ill-conditioning challenges due to structural ambiguities. This study aims to address these issues by integrating the implicit anatomical understanding of vision foundation models (FMs) into a multi-scale unsupervised framework for accurate and robust DIR. Approach. Our method takes moving and fixed images as inputs and leverages a pre-trained encoder from a vision FM to extract latent features. These features are merged with those extracted by convolutional adaptors to incorporate inductive bias. Correlation-aware multi-layer perceptrons decode the features into DVFs. A pyramid architecture is implemented to capture multi-range dependencies, further enhancing the DIR robustness and accuracy. We evaluated our method using a multi-modality, cross-institutional database consisting of 150 cardiac cine MR and 40 liver CT. Main results. Our model generates realistic and accurate DVFs. Moving images deformed by our method showed excellent similarity to fixed images, achieving a registration Dice score of 0.869 ± 0.093 for cardiac MRI and an average landmark error of 1.60 ± 1.44 mm for liver CT, substantially surpassing the state-of-the-art methods. Ablation studies further verified the effectiveness of integrating foundation features to improve DIR accuracy ( p < 0.05). Significance. Our novel approach demonstrates significant advancements in DIR for multi-modality images with complex structures and low contrasts, making it a powerful tool for a wide range of applications in medical image analysis.
25 September 2025 in Physics in Medicine & Biology
Objective. To overcome the limitations of electroacoustic tomography (EAT) in clinical settings-particularly the artifacts and distortions caused by limited-angle data acquisition-and enable accurate, efficient visualization of electric field distributions for electroporation-based therapies. Approach. We developed a deep learning-based framework that enhances 3D EAT image reconstruction from single-view projections by leveraging the large foundation model (LFM) SAM-Med3D. The encoder was modified into a local-global feature fusion architecture that extracts multi-scale features from intermediate transformer layers, preserving fine structural details while maintaining computational efficiency. A lightweight decoder with progressive up-sampling and skip connections was employed to generate high-resolution images, addressing limitations of conventional U-Net architectures. Main results. We collected a dataset of 50 EAT scans-each with 120 views-for a total of 6000 views. These were acquired from water phantoms and tissue samples under varied electrode configurations and voltages, and split into training (30 scans, 3600 views), validation (10 scans, 1200 views), and testing (10 scans, 1200 views). Our model significantly outperformed baseline 3D U-Nets, achieving an RMSE of 0.0092, PSNR of 41.10, and SSIM of 0.9377. Remarkably, our method reconstructs a full-view 3D EAT image from a single view in just 2 s, demonstrating its potential for near real-time monitoring and adaptive dose verification in electroporation-based therapies. Significance. This is the first application of a LFM like SAM-Med3D for enhancing 3D EAT imaging. The proposed framework addresses the critical challenge of limited-angle data in EAT and demonstrates strong potential for improving precision and safety in electroporation-based therapies, thereby advancing the technique’s clinical viability.
13 March 2026 in Radiation Medicine and Protection