PurposeCochlear implant (CI) surgery treats severe hearing loss by inserting an electrode array into the cochlea to stimulate the auditory nerve. An important step in this procedure is mastoidectomy, which removes part of the mastoid region of the temporal bone to provide surgical access. Accurate mastoidectomy shape prediction from preoperative imaging improves presurgical planning, reduces risks, and enhances surgical outcomes. Despite its importance, there are limited deep-learning-based studies regarding this topic due to the challenges of acquiring ground-truth labels. We address this gap by investigating self-supervised and weakly-supervised learning models to predict the mastoidectomy region without human annotations.ApproachWe propose a hybrid self-supervised and weakly-supervised learning framework to predict the mastoidectomy region directly from preoperative CT scans, where the mastoid remains intact. Our self-supervised learning approach reconstructs the postmastoidectomy 3D surface from preoperative imaging, aiming to align with the corresponding intraoperative microscope views for future surgical navigation-related applications. Postoperative CT scans are used in the self-supervised learning model to assist training procedures despite additional challenges such as metal artifacts and low signal-to-noise ratios introduced by them. To further improve the accuracy and robustness, we introduce a Mamba-based weakly-supervised model that refines mastoidectomy shape prediction by using 3D T-distribution loss function, inspired by the student-t distribution. Weak supervision is achieved by leveraging segmentation results from the prior self-supervised framework, eliminating the manual data labeling process.ResultsOur hybrid method achieves a mean Dice score of 0.72 when predicting the complex and boundary-less mastoidectomy shape, surpassing state-of-the-art approaches and demonstrating strong performance. The method provides groundwork for constructing 3D postmastoidectomy surfaces directly from the corresponding preoperative CT scans.ConclusionTo our knowledge, this is the first work that integrates self-supervised and weakly-supervised learning for mastoidectomy shape prediction, offering a robust and efficient solution for CI surgical planning while leveraging 3D T-distribution loss in weakly-supervised medical imaging.

With advancements in AI, appearance-based gaze estimation methods have improved benchmark performance, but practical deployment still lags behind commercial eye-trackers. Factors like model size, inference time, and privacy often go unaddressed. Meanwhile, webcam-based eye-tracking methods remain sensitive to head movement. To tackle these issues, we introduce WebEyeTrack, a browser-native framework that integrates a novel lightweight gaze model, metric head pose estimation, and on-device few-shot learning with as few as nine per-user calibration samples (\(k \le 9\)). WebEyeTrack adapts to new users, achieving competitive performance with an error margin of 2.32 cm on GazeCapture and real-time inference latency of 2.4 milliseconds on an iPhone 14. Our open-source code is available at github.com/RedForestAI/WebEyeTrack.

This study presents a novel framework for 3D gaze tracking tailored for mixed-reality settings, aimed at enhancing joint attention and collaborative efforts in team-based scenarios. Conventional gaze tracking, often limited by monocular cameras and traditional eye-tracking apparatus, struggles with simultaneous data synchronization and analysis from multiple participants in group contexts. Our proposed framework leverages state-of-the-art computer vision and machine learning techniques to overcome these obstacles, enabling precise 3D gaze estimation without dependence on specialized hardware. Utilizing facial recognition and deep learning, the framework achieves real-time, tracking of gaze patterns across several individuals, addressing common depth estimation errors, and ensuring spatial and identity consistency within the dataset. This provides mechanisms for significant advances in behavior and interaction analysis in educational and professional training applications in dynamic and unstructured environments. For transparency and to promote further development, we released the code at https://github.com/edavalosanaya/3DGazeTracking_ICMIW2024.

Eye-tracking offers rich insights into student cognition and engagement, but remains underutilized in classroom-facing educational technology due to challenges in data interpretation and accessibility. In this paper, we present the iterative design and evaluation of a gaze-based learning analytics dashboard for English Language Arts (ELA), developed through five studies involving teachers and students. Guided by user-centered design and data storytelling principles, we explored how gaze data can support reflection, formative assessment, and instructional decision-making. Our findings demonstrate that gaze analytics can be approachable and pedagogically valuable when supported by familiar visualizations, layered explanations, and narrative scaffolds. We further show how a conversational agent, powered by a large language model (LLM), can lower cognitive barriers to interpreting gaze data by enabling natural language interactions with multimodal learning analytics. We conclude with design implications for future EdTech systems that aim to integrate novel data modalities in classroom contexts.