Abdominal aortic aneurysm (AAA) is a fatal clinical condition with high mortality. Computed tomography angiography (CTA) imaging is the preferred minimally invasive modality for the long-term postoperative observation of AAA. Accurate segmentation of the thrombus region of interest (ROI) in a postoperative CTA image volume is essential for quantitative assessment and rapid clinical decision making by clinicians. Few investigators have proposed the adoption of convolutional neural networks (CNN). Although these methods demonstrated the potential of CNN architectures by automating the thrombus ROI segmentation, the segmentation performance can be further improved. The existing methods performed the segmentation process independently per 2D image and were incapable of using adjacent images, which could be useful for the robust segmentation of thrombus ROIs. In this work, we propose a thrombus ROI segmentation method to utilize not only the spatial features of a target image, but also the volumetric coherence available from adjacent images. We newly adopted a recurrent neural network, bi-directional convolutional long short-term memory (Bi-CLSTM) architecture, which can learn coherence between a sequence of data. This coherence learning capability can be useful for challenging situations, for example, when the target image exhibits inherent postoperative artifacts and noises, the inclusion of adjacent images would facilitate learning more robust features for thrombus ROI segmentation. We demonstrate the segmentation capability of our Bi-CLSTM-based method with a comparison of the existing 2D-based thrombus ROI segmentation counterpart as well as other established 2D- and 3D-based alternatives. Our comparison is based on a large-scale clinical dataset of 60 patient studies (i.e., 60 CTA image volumes). The results suggest the superior segmentation performance of our Bi–CLSTM-based method by achieving the highest scores of the evaluation metrics, e.g., our Bi-CLSTM results were 0.0331 higher on total overlap and 0.0331 lower on false negative when compared to 2D U-net++ as the second-best.

Volume reconstruction techniques are gaining increasing interest in medical domains due to their potential to learn complex 3D structural information from sparse 2D images. Recently, neural radiance fields (NeRF), which implicitly model continuous radiance fields based on multi-layer perceptrons to enable volume reconstruction of objects at arbitrary resolution, have gained traction in natural image volume reconstruction. However, the direct application of NeRF to medical volume reconstruction presents unique challenges due to differences in imaging principles, internal structure requirements, and boundary delineation. In this paper, we evaluate different NeRF techniques developed for natural images, including sampling strategies, feature encoding, and the use of complimentary features, by applying them to medical images. We evaluate three state-of-the-art NeRF techniques on four datasets of medical images of different complexity. Our goal is to identify the strengths, limitations, and future directions for integrating NeRF into the medical domain.

Direct volume rendering (DVR) enables the visual exploration of regions of interest (ROI) in a volume in an interactive manner. Mixed reality head mounted devices (MR-HMDs) have the potential to expand the utility of DVR. MR-HMDs enable not only the high-quality and stereoscopic DVR but also its in-situ visualizations of complementary data to assist in the real-world tasks of users. The state-of-the-art MR-HMD works, however, mainly focus on investigating the feasibility and effectiveness of DVR to target MR applications. The interactive ROI-based exploration of DVR has been severely limited in prior research, and user optimization is still confined to the primitive image manipulation of pre-configured ROIs (e.g., view rotation). In this work, we propose an interactive and semi-automated exploration approach for ROI-based DVRs in MR-HMDs by introducing a new MR-HMD-based rendering parameters (i.e., transfer functions (TFs)) optimization and utilizing the built-in biometric interfaces. Using eye-gazing and hand gestures, users can select ROIs within an initial DVR visually and interactively. Our MR-TF approach (named hereafter) then automatically optimizes the corresponding TF toward ensuring the visibility of ROIs. Additionally, our MR-TF approach addresses a critical rendering issue in MR-HMDs, i.e., the color perception of ROIs should be consistent regardless of the variations of intermixing physical backgrounds. Our MR-TF approach quantifies the color perception and integrates to the TF optimization for enhanced ROIs. We achieve on-device implementation of our MR-TF approach on the commercial MR-HMD with low computing capability, Microsoft HoloLens 2. Our evaluations with different types of volumes demonstrate the capabilities and practicality of our MR-TF approach.

Paranasal sinus surgery, a common treatment for chronic rhinosinusitis, requires exceptional precision due to the proximity of critical anatomical structures. To ensure accurate instrument control and clear visualization of the surgical site, surgeons utilize computer-aided surgical navigation (CSN). A key component of CSN is the registration process, which is traditionally reliant on manual or marker-based techniques. However, there is a growing shift toward marker-less registration methods. In previous work, we investigated a mesh-based registration approach using a Mixed Reality Head-Mounted Display (MR-HMD), specifically the Microsoft HoloLens 2. However, this method faced limitations, including depth holes and invalid values. These issues stemmed from the device’s low-resolution camera specifications and the 3D projection steps required to upscale RGB camera spaces. In this study, we propose a novel automated marker-less registration method leveraging Neural Radiance Field (NeRF) technology with an MR-HMD. To address insufficient depth information in the previous approach, we utilize rendered-depth images generated by the trained NeRF model. We evaluated our method against two other techniques, including prior mesh-based registration, using a facial phantom and three participants. The results demonstrate our proposed method achieves at least a 0.873 mm (12%) improvement in registration accuracy compared to others.

Mixed Reality (MR) techniques begin to penetrate our daily life. Due to the features of MR techniques to seamlessly intermix a user’s real environment with useful virtual contents in a natural manner, their potential applications range from information visualization and sharing and remote work collaboration to education and training in various fields. However, due to the steep learning curve in content creation and development, it is not trivial for novice users to create MR contents by themselves. In this work, we propose “Codeless Content Creator System”, an MR system that helps novice users rapidly and conveniently create MR contents using MR devices rather than relying on complicated software development tools such as Unity. A simple user study found that our system is useful and convenient for the MR contents creation tasks.