1 November 2019 in IEEE Transactions on Medical Imaging
We present a self-supervised approach to training convolutional neural networks for dense depth estimation from monocular endoscopy data without a priori modeling of anatomy or shading. Our method only requires monocular endoscopic videos and a multi-view stereo method, e.g., structure from motion, to supervise learning in a sparse manner. Consequently, our method requires neither manual labeling nor patient computed tomography (CT) scan in the training and application phases. In a cross-patient experiment using CT scans as groundtruth, the proposed method achieved submillimeter mean residual error. In a comparison study to recent self-supervised depth estimation methods designed for natural video on in vivo sinus endoscopy data, we demonstrate that the proposed approach outperforms the previous methods by a large margin. The source code for this work is publicly available online at https://github.com/lppllppl920/EndoscopyDepthEstimation-Pytorch.
7 August 2023 in IEEE Transactions on Medical Imaging
Image-based 2D/3D registration is a critical technique for fluoroscopic guided surgical interventions. Conventional intensity-based 2D/3D registration approaches suffer from a limited capture range due to the presence of local minima in hand-crafted image similarity functions. In this work, we aim to extend the 2D/3D registration capture range with a fully differentiable deep network framework that learns to approximate a convex-shape similarity function. The network uses a novel Projective Spatial Transformer (ProST) module that has unique differentiability with respect to 3D pose parameters, and is trained using an innovative double backward gradient-driven loss function. We compare the most popular learning-based pose regression methods in the literature and use the well-established CMAES intensity-based registration as a benchmark. We report registration pose error, target registration error (TRE) and success rate (SR) with a threshold of 10mm for mean TRE. For the pelvis anatomy, the median TRE of ProST followed by CMAES is 4.4mm with a SR of 65.6% in simulation, and 2.2mm with a SR of 73.2% in real data. The CMAES SRs without using ProST registration are 28.5% and 36.0% in simulation and real data, respectively. Our results suggest that the proposed ProST network learns a practical similarity function, which vastly extends the capture range of conventional intensity-based 2D/3D registration. We believe that the unique differentiable property of ProST has the potential to benefit related 3D medical imaging research applications.
9 August 2022 in Thyroid®
Background: Recent studies have reported a wide spectrum of ocular surface injuries in the context of autoimmune reactions in Graves' orbitopathy (GO). Increased expression of inflammatory mediators in tears of GO patients suggests that the lacrimal glands could be a target for immune responses. However, the immunophenotype for GO lacrimal microenvironment is not known. This study aimed to elucidate the pathological changes of GO lacrimal glands. Methods: In this case–control study, lacrimal glands were surgically collected from GO patients who underwent orbital decompression surgery and control subjects who underwent other ocular-related surgery. Bulk RNA-sequencing, flow cytometry with dimensional reduction, and immunohistochemical and multiplexed stainings were conducted. Western blotting and multipathway assays were performed in CD34 + fibroblasts derived from lacrimal and orbital tissues. Results: Increased expression of cytokines and chemokines accompanied by a variety of immune cell infiltrations mainly involving T cells, B cells, and monocytes was found in GO lacrimal glands. An in-depth investigation into T cell subsets revealed interferon (IFN)-γ-producing T helper (Th)1 and interleukin (IL)-17A-producing Th17 cell-dominated autoimmunity in the active GO lacrimal microenvironment. Both fibrosis and adipogenesis were observed in GO lacrimal tissue remodeling. IL-17A, not IFN-γ, stimulated transforming growth factor-β-initiated myofibroblast differentiation as well as 15-deoxy-Δ 12,14 -prostaglandin J 2 -initiated adipocyte differentiation in CD34 + lacrimal fibroblasts (LFs) and orbital fibroblasts (OFs), respectively. IL-17A activated many fibrotic and adipogenic-related signaling pathways in CD34 + LFs and OFs. A novel anti-IL-17A monoclonal antibody SHR-1314 could reverse the promoting effect of IL-17A on fibrosis and adipogenesis in CD34 + LFs and OFs. Conclusions: Our findings provide evidence for the infiltration of different lymphocytes into GO lacrimal microenvironment, where Th1 and Th17 cells characterize the onset of active lacrimal inflammation and contribute to tissue remodeling. These findings may have potential future therapeutic implications regarding the utility of anti-IL-17A therapy, which should be studied in future research.