Remarkable progress has been made in image recognition, primarily due to the availability of large-scale annotated datasets and deep convolutional neural networks (CNNs). CNNs enable learning data-driven, highly representative, hierarchical image features from sufficient training data. However, obtaining datasets as comprehensively annotated as ImageNet in the medical imaging domain remains a challenge. There are currently three major techniques that successfully employ CNNs to medical image classification: training the CNN from scratch, using off-the-shelf pre-trained CNN features, and conducting unsupervised CNN pre-training with supervised fine-tuning. Another effective method is transfer learning, i.e., fine-tuning CNN models pre-trained from natural image dataset to medical image tasks. In this paper, we exploit three important, but previously understudied factors of employing deep convolutional neural networks to computer-aided detection problems. We first explore and evaluate different CNN architectures. The studied models contain 5 thousand to 160 million parameters, and vary in numbers of layers. We then evaluate the influence of dataset scale and spatial image context on performance. Finally, we examine when and why transfer learning from pre-trained ImageNet (via fine-tuning) can be useful. We study two specific computer-aided detection (CADe) problems, namely thoraco-abdominal lymph node (LN) detection and interstitial lung disease (ILD) classification. We achieve the state-of-the-art performance on the mediastinal LN detection, and report the first five-fold cross-validation classification results on predicting axial CT slices with ILD categories. Our extensive empirical evaluation, CNN model analysis and valuable insights can be extended to the design of high performance CAD systems for other medical imaging tasks.
Objective:To validate a fully automated CT-based spinal trabecular bone mineral density (BMD) tool and apply it to a longitudinal screening cohort.Methods:The automated BMD tool was retrospectively...
Objective:To investigate a fully automated CT-based adiposity tool, applying it to a longitudinal adult screening cohort.Methods:A validated automated adipose tissue segmentation algorithm was appl...
15 November 2017 in IEEE Transactions on Medical Imaging
Tumor growth is associated with cell invasion and mass-effect, which are traditionally formulated by mathematical models, namely reaction-diffusion equations and biomechanics. Such models can be personalized based on clinical measurements to build the predictive models for tumor growth. In this paper, we investigate the possibility of using deep convolutional neural networks to directly represent and learn the cell invasion and mass-effect, and to predict the subsequent involvement regions of a tumor. The invasion network learns the cell invasion from information related to metabolic rate, cell density, and tumor boundary derived from multimodal imaging data. The expansion network models the mass-effect from the growing motion of tumor mass. We also study different architectures that fuse the invasion and expansion networks, in order to exploit the inherent correlations among them. Our network can easily be trained on population data and personalized to a target patient, unlike most previous mathematical modeling methods that fail to incorporate population data. Quantitative experiments on a pancreatic tumor data set show that the proposed method substantially outperforms a state-of-the-art mathematical model-based approach in both accuracy and efficiency, and that the information captured by each of the two subnetworks is complementary.
Pancreatic neuroendocrine tumors are abnormal growths of hormone-producing cells in the pancreas. Unlike the brain which is protected by the skull, the pancreas can be significantly deformed by its surrounding organs. Consequently, the tumor shape differences observable from images at different time points arise from both tumor growth and pancreatic motion, and tumor growth model personalization may be compromised if such motion is ignored. Therefore, we incorporate pancreatic motion information derived from deformable image registration in model personalization. For more accurate mechanical interactions between tumor growth and pancreatic motion, elastic-growth decomposition is used with a hyperelastic constitutive law to model the mass effect, which allows growth modeling while conserving the mechanical properties. Furthermore, a way of coupling the finite difference method and the finite element method is proposed to greatly reduce the computation time. With both 2-[ 18 F]-fluoro-2-deoxy-D-glucose positron emission tomographic and contrast-enhanced computed tomographic images, functional, structural, and motion data are combined for a patient-specific model. Experiments on synthetic and clinical data show the importance of image-derived motion on estimating pathophysiologically plausible mechanical properties and the promising performance of our framework. From seven patient data sets, the recall, precision, Dice coefficient, relative volume difference, and average surface distance between the personalized tumor growth simulations and the measurements were 83.2 ±8.8%, 86.9 ±8.3%, 84.4 ±4.0%, 13.9 ±9.8%, and 0.6 ±0.1 mm, respectively.