The automatic diagnosis of various retinal diseases from fundus images is important to support clinical decision-making. However, developing such automatic solutions is challenging due to the requirement of a large amount of human-annotated data. Recently, unsupervised/self-supervised feature learning techniques receive a lot of attention, as they do not need massive annotations. Most of the current self-supervised methods are analyzed with single imaging modality and there is no method currently utilize multi-modal images for better results. Considering that the diagnostics of various vitreoretinal diseases can greatly benefit from another imaging modality, e.g., FFA, this paper presents a novel self-supervised feature learning method by effectively exploiting multi-modal data for retinal disease diagnosis. To achieve this, we first synthesize the corresponding FFA modality and then formulate a patient feature-based softmax embedding objective. Our objective learns both modality-invariant features and patient-similarity features. Through this mechanism, the neural network captures the semantically shared information across different modalities and the apparent visual similarity between patients. We evaluate our method on two public benchmark datasets for retinal disease diagnosis. The experimental results demonstrate that our method clearly outperforms other self-supervised feature learning methods and is comparable to the supervised baseline. Our code is available at GitHub1.
Cherenkov emission induced by external beam radiation from a clinical linear accelerator has been shown in preclinical molecular imaging and clinical imaging. The broad spectrum Cherenkov emission should have a short wavelength infrared (SWIR, 1000–1700 nm) component, as predicted theoretically. To the best of our knowledge, this Letter is the first experimental observation of this SWIR Cherenkov emission induced by external beam radiation. The measured spectrum of SWIR Cherenkov emission matches the theoretical prediction, with a fluence rate near one-third of the visible and near-infrared red emissions (Vis-NIR, 400–900 nm). Imaging in water-based phantoms and biological tissues indicates that there is a sufficient fluence rate for radiotherapy dosimetry applications. The spatial resolution is improved approximately 5.3 times with SWIR Cherenkov emission detection versus Vis-NIR Cherenkov emission, which provides some improvement in the potential for higher resolution Cherenkov emission dosimetry and molecular sensing during clinical radiotherapy by imaging with SWIR wavelengths.
Significance: Deep-tissue penetration by x-rays to induce optical responses of specific molecular reporters is a new way to sense and image features of tissue function in vivo. Advances in this field are emerging, as biocompatible probes are invented along with innovations in how to optimally utilize x-ray sources. Aim: A comprehensive review is provided of the many tools and techniques developed for x-ray-induced optical molecular sensing, covering topics ranging from foundations of x-ray fluorescence imaging and x-ray tomography to the adaptation of these methods for sensing and imaging in vivo. Approach: The ways in which x-rays can interact with molecules and lead to their optical luminescence are reviewed, including temporal methods based on gated acquisition and multipoint scanning for improved lateral or axial resolution. Results: While some known probes can generate light upon x-ray scintillation, there has been an emergent recognition that excitation of molecular probes by x-ray-induced Cherenkov light is also possible. Emission of Cherenkov radiation requires a threshold energy of x-rays in the high kV or MV range, but has the advantage of being able to excite a broad range of optical molecular probes. In comparison, most scintillating agents are more readily activated by lower keV x-ray energies but are composed of crystalline inorganic constituents, although some organic biocompatible agents have been designed as well. Methods to create high-resolution structured x-ray-optical images are now available, based upon unique scanning approaches and/or a priori knowledge of the scanned x-ray beam geometry. Further improvements in spatial resolution can be achieved by careful system design and algorithm optimization. Current applications of these hybrid x-ray-optical approaches include imaging of tissue oxygenation and pH as well as of certain fluorescent proteins. Conclusions: Discovery of x-ray-excited reporters combined with optimized x-ray scan sequences can improve imaging resolution and sensitivity.
Cone-beam x-ray luminescence computed tomography (CB-XLCT) has been proposed as a promising imaging tool, which enables three-dimensional imaging of the distribution of nanophosphors (NPs) in small animals. However, the reconstruction performance is usually unsatisfactory in terms of spatial resolution due to the ill-posedness of the CB-XLCT inverse problem. To alleviate this problem and to achieve high spatial resolution, a reconstruction method consisting of inner and outer iterations based on a restarted strategy is proposed. In this method, the primal–dual Newton conjugate gradient method (pdNCG) is adopted in the inner iterations to get fast reconstruction, which is used for resetting the permission region and increasing the convergence speed of the outer iteration. To assess the performance of the method, both numerical simulation and physical phantom experiments were conducted with a CB-XLCT system. The results demonstrate that compared with conventional reconstruction methods, the proposed re-pdNCG method can accurately and efficiently resolve the adjacent NPs with the least relative error.
Cherenkov emission induced by external beam radiation therapy from a clinical linear accelerator (LINAC) can be used to excite phosphors deep in biological tissues. As with all luminescence imaging, there is a desire to minimize the spectral overlap between the excitation light and emission wavelengths, here between the Cherenkov and the phosphor. Cherenkov excited short-wavelength infrared (SWIR, 1000 to 1700 nm) fluorescence imaging has been demonstrated for the first time, using long Stokes-shift fluorophore PdSe quantum dots (QD) with nanosecond lifetime and an optimized SWIR detection. The 1 / λ2 intensity spectrum characteristic of Cherenkov emission leads to low overlap of this into the fluorescence spectrum of PdSe QDs in the SWIR range. Additionally, using a SWIR camera itself inherently ignores the stronger Cherenkov emission wavelengths dominant across the visible spectrum. The SWIR luminescence was shown to extend the depth sensitivity of Cherenkov imaging, which could be used for applications in radiotherapy sensing and imaging in human tissue with targeted molecular probes.