Throughout the course of delivering a radiation therapy treatment, which may take several weeks, a patient’s anatomy may change drastically, and adaptive radiation therapy (ART) may be needed. Cone-beam computed tomography (CBCT), which is often available during the treatment process, can be used for both patient positioning and ART re-planning. However, due to the prominent amount of noise, artifacts, and inaccurate Hounsfield unit (HU) values, the dose calculation based on CBCT images could be inaccurate for treatment planning. One way to solve this problem is to convert CBCT images to more accurate synthesized CT (sCT) images. In this work, we have developed a cycle-consistent generative adversarial network framework (CycleGAN) to synthesize CT images from CBCT images. This model is capable of image-to-image translation using unpaired CT and CBCT images in an unsupervised learning setting. The sCT images generated from CBCT through this CycleGAN model are visually and quantitatively similar to real CT images with decreased mean absolute error (MAE) from 69.29 HU to 29.85 HU for head-and-neck (H&N) cancer patients. The dose distributions calculated on the sCT by CycleGAN demonstrated a higher accuracy than those on CBCT in a 3D gamma index analysis with increased gamma index pass rate from 86.92% to 96.26% under 1 mm/1% criteria, when using the deformed planning CT image (dpCT) as the reference. We also compared the CycleGAN model with other unsupervised learning methods, including deep convolutional generative adversarial networks (DCGAN) and progressive growing of GANs (PGGAN), and demonstrated that CycleGAN outperformed the other two models. A phantom study has been conducted to compare sCT with dpCT, and the increase of structural similarity index from 0.91 to 0.93 shows that CycleGAN performed better than DIR in terms of preserving anatomical accuracy.

Recently, the diffusion model has emerged as a superior generative model that can produce high quality and realistic images. However, for medical image translation, the existing diffusion models are deficient in accurately retaining structural information since the structure details of source domain images are lost during the forward diffusion process and cannot be fully recovered through learned reverse diffusion, while the integrity of anatomical structures is extremely important in medical images. For instance, errors in image translation may distort, shift, or even remove structures and tumors, leading to incorrect diagnosis and inadequate treatments. Training and conditioning diffusion models using paired source and target images with matching anatomy can help. However, such paired data are very difficult and costly to obtain, and may also reduce the robustness of the developed model to out-of-distribution testing data. We propose a frequency-guided diffusion model (FGDM) that employs frequency-domain filters to guide the diffusion model for structure-preserving image translation. Based on its design, FGDM allows zero-shot learning, as it can be trained solely on the data from the target domain, and used directly for source-to-target domain translation without any exposure to the source-domain data during training. We trained FGDM solely on the head-and-neck CT data, and evaluated it on both head-and-neck and lung cone-beam CT (CBCT)-to-CT translation tasks. FGDM outperformed the state-of-the-art methods (GAN-based, VAE-based, and diffusion-based) in metrics of Fréchet Inception Distance (FID), Peak Signal-to-Noise Ratio (PSNR), and Structural Similarity Index Measure (SSIM), showing its significant advantages in zero-shot medical image translation.

A number of image-processing problems can be formulated as optimization problems. The objective function typically contains several terms specifically designed for different purposes. Parameters in front of these terms are used to control the relative importance among them. It is of critical importance to adjust these parameters, as quality of the solution depends on their values. Tuning parameters are a relatively straight forward task for a human, as one can intuitively determine the direction of parameter adjustment based on the solution quality. Yet manual parameter tuning is not only tedious in many cases, but also becomes impractical when a number of parameters exist in a problem. Aiming at solving this problem, this paper proposes an approach that employs deep reinforcement learning to train a system that can automatically adjust parameters in a human-like manner. We demonstrate our idea in an example problem of optimization-based iterative computed tomography (CT) reconstruction with a pixel-wise total-variation regularization term. We set up a parameter-tuning policy network (PTPN), which maps a CT image patch to an output that specifies the direction and amplitude by which the parameter at the patch center is adjusted. We train the PTPN via an end-to-end reinforcement learning procedure. We demonstrate that under the guidance of the trained PTPN, reconstructed CT images attain quality similar or better than those reconstructed with manually tuned parameters.

Lymph node metastasis (LNM) is a significant prognostic factor in patients with head and neck cancer, and the ability to predict it accurately is essential to optimizing treatment. Positron emission tomography (PET) and computed tomography (CT) imaging are routinely used to identify LNM. Although large or highly active lymph nodes (LNs) have a high probability of being positive, identifying small or less reactive LNs is challenging. The accuracy of LNM identification strongly depends on the physician’s experience, so an automatic prediction model for LNM based on CT and PET images is warranted to assist LMN identification across care providers and facilities. Radiomics and deep learning are the two promising imaging-based strategies for node malignancy prediction. Radiomics models are built based on handcrafted features, while deep learning learns the features automatically. To build a more reliable model, we proposed a hybrid predictive model that takes advantages of both radiomics and deep learning based strategies. We designed a new many-objective radiomics (MaO-radiomics) model and a 3D convolutional neural network (3D-CNN) that fully utilizes spatial contextual information, and we fused their outputs through an evidential reasoning (ER) approach. We evaluated the performance of the hybrid method for classifying normal, suspicious and involved LNs. The hybrid method achieves an accuracy (ACC) of 0.88 while XmasNet and Radiomics methods achieve 0.81 and 0.75, respectively. The hybrid method provides a more accurate way for predicting LNM using PET and CT.

Cervical tumor segmentation on 3D ¹⁸ FDG PET images is a challenging task because of the proximity between cervix and bladder, both of which can uptake ¹⁸ FDG tracers. This problem makes traditional segmentation based on intensity variation methods ineffective and reduces overall accuracy. Based on anatomy knowledge, including ‘roundness’ of the cervical tumor and relative positioning between the bladder and cervix, we propose a supervised machine learning method that integrates convolutional neural network (CNN) with this prior information to segment cervical tumors. First, we constructed a spatial information embedded CNN model (S-CNN) that maps the PET image to its corresponding label map, in which bladder, other normal tissue, and cervical tumor pixels are labeled as −1, 0, and 1, respectively. Then, we obtained the final segmentation from the output of the network by a prior information constrained (PIC) thresholding method. We evaluated the performance of the PIC-S-CNN method on PET images from 50 cervical cancer patients. The PIC-S-CNN method achieved a mean Dice similarity coefficient (DSC) of 0.84 while region-growing, Chan-Vese, graph-cut, fully convolutional neural networks (FCN) based FCN-8 stride, and FCN-2 stride, and U-net achieved 0.55, 0.64, 0.67, 0.71, 0.77, and 0.80 mean DSC, respectively. The proposed PIC-S-CNN provides a more accurate way for segmenting cervical tumors on 3D PET images. Our results suggest that combining deep learning and anatomic prior information may improve segmentation accuracy for cervical tumors.

There can be significant uncertainty when identifying cervical lymph node (LN) metastases in patients with oropharyngeal squamous cell carcinoma (OPSCC) despite the use of modern imaging modalities such as positron emission tomography (PET) and computed tomography (CT) scans. Grossly involved LNs are readily identifiable during routine imaging, but smaller and less PET-avid LNs are harder to classify. We trained a convolutional neural network (CNN) to detect malignant LNs in patients with OPSCC and used quantitative measures of uncertainty to identify the most reliable predictions. Our dataset consisted of images of 791 LNs from 129 patients with OPSCC who had preoperative PET/CT imaging and detailed pathological reports after neck dissections. These LNs were segmented on PET/CT imaging and then labeled according to the pathology reports. An AlexNet-like CNN was trained to classify LNs as malignant or benign. We estimated epistemic and aleatoric uncertainty by using dropout variational inference and test-time augmentation, respectively. CNN performance was stratified according to the median epistemic and aleatoric uncertainty values calculated using the validation cohort. Our model achieved an area under the receiver operating characteristic (ROC) curve (AUC) of 0.99 on the testing dataset. Sensitivity and specificity were 0.94 and 0.90, respectively. Epistemic and aleatoric uncertainty values were statistically larger for false negative and false positive predictions than for true negative and true positive predictions (p < 0.001). Model sensitivity and specificity were 1.0 and 0.98, respectively, for cases with epistemic uncertainty lower than the median value of the incorrect predictions in the validation dataset. For cases with higher epistemic uncertainty, sensitivity and specificity were 0.67 and 0.41, respectively. Model sensitivity and specificity were 1.0 and 0.98, respectively, for cases with aleatoric uncertainty lower than the median value of the incorrect predictions in the validation dataset. For cases with higher aleatoric uncertainty, sensitivity and specificity were 0.67 and 0.37, respectively. We used a CNN to predict the malignant status of LNs in patients with OPSCC with high accuracy, and we showed that uncertainty can be used to quantify a prediction’s reliability. Assigning measures of uncertainty to predictions could improve the accuracy of LN classification by efficiently identifying instances where expert evaluation is needed to corroborate a model’s prediction.