Alzheimer's Disease (AD) is a neurodegenerative disorder that causes a continuous decline in cognitive functions and eventually results in death. An early AD diagnosis is important for taking active measures to slow its deterioration. Traditional diagnoses are usually based on clinical experience, which is limited by several realistic factors. In this paper, we focus on exploiting deep learning techniques to diagnose AD based on eye-tracking behaviors. Visual attention, as a typical eye-tracking behavior, is of great clinical value in detecting cognitive abnormalities in AD patients. To better analyze the differences in visual attention between AD patients and normals, we first conducted a 3D comprehensive visual task on a noninvasive eye-tracking system to collect visual attention heatmaps. Then a multilayered comparison convolutional neural network (MC-CNN) is proposed to distinguish the visual attention differences between AD patients and normals. In MC-CNN, the multilayered feature representations of heatmaps were obtained by hierarchical residual blocks to better encode eye movement behaviors, which were further integrated into a distance vector to benefit the comprehensive visual task. From evaluation, MC-CNN can distinguish AD patients from normals with 0.84 accuracy, 0.86 recall, 0.82 precision, 0.83 F1-score and 0.90 area under the curve (AUC). The above results demonstrate the effectiveness of the proposed MC-CNN in AD diagnosis based on the comprehensive 3D visual task.

Alzheimer’s disease (AD) is a degenerative neurological disease that occurs in the elderly with typical symptoms of decline in cognition, manifested by eye movement behaviors. The key to AD treatment requires early detection of cognitive impairment, which relies on frequent medical screening. This article proposes an Internet of Things (IoT) architecture constructed with eye-tracker (ET) nodes and cloud-based diagnosis enabled by machine learning (ML), which can provide convenient screening of oculomotor abnormalities and automatic identification of early-stage AD. The bespoke ET nodes collect 3-D oculomotor responses from diverse stereo video stimulation trials and transmit data into a dedicated multimodal ML (MMML) algorithm in the cloud. The algorithm incorporates multimodal features extracted from diverse oculomotor types to enhance the classification accuracy and optimized data dimension reduction in feature fusion to improve the classifier’s performance. From evaluation, the proposed method can distinguish AD patients from the control group with 86% accuracy (ACC), 78% true positive rate (TPR), and 90% positive predictive value (PPV). The results confirm the effectiveness of our MMML algorithm in AD diagnosis with the fusion of multimodal oculomotor features and prove the feasibility of our IoT-powered eye-tracking solution for AD screening.

As a powerful global registration method for point clouds, the 4-points congruent sets (4PCS) algorithm has been wildly used in the 3D scene reconstruction field. In this paper, we propose an adaptive 4PCS (A4PCS), which aims to provide a robustness rigid transformation for two or more overlapping laser scans. The proposed method only incorporates the distance information of the stereoscopic base set and a fast mechanism for congruent base extraction into 4PCS. To ensure the registration accuracy when dealing with restrictive situations, such as point clouds with small overlaps or scenes with symmetrical structures, a non-coplanar 4-points base set is adopted without extra time consumption. Besides, the adaptive set fine-tuning is introduced to the point pair searching process to accelerate the convergence of the algorithm. In addition, we replace the binary cost function of the original 4PCS with a modified estimator to strengthen the robustness of the proposed method. Experiments on thirteen pairs of point clouds for 3D indoor scenes, including ten regular size models and three scenes of one large-scale model, can demonstrate the accuracy and efficiency of the proposed method.

Pedestrian navigation systems could serve as a good supplement for other navigation methods or for extending navigation into areas where other navigation systems are invalid. Due to the accumulation of inertial sensing errors, foot-mounted inertial-sensor-based pedestrian navigation systems (PNSs) suffer from drift, especially heading drift. To mitigate heading drift, considering the complexity of human motion and the environment, we introduce a novel hybrid framework that integrates a foot-state classifier that triggers the zero-velocity update (ZUPT) algorithm, zero-angular-rate update (ZARU) algorithm, and a state lock, a magnetic disturbance detector, a human-motion-classifier-aided adaptive fusion module (AFM) that outputs an adaptive heading error measurement by fusing heuristic and magnetic algorithms rather than simply switching them, and an error-state Kalman filter (ESKF) that estimates the optimal systematic error. The validation datasets include a Vicon loop dataset that spans 324.3 m in a single room for approximately 300 s and challenging walking datasets that cover large indoor and outdoor environments with a total distance of 12.98 km. A total of five different frameworks with different heading drift correction methods, including the proposed framework, were validated on these datasets, which demonstrated that our proposed ZUPT–ZARU–AFM–ESKF-aided PNS outperforms other frameworks and clearly mitigates heading drift.

Eye-tracking technology is increasingly used in applications such as 3D displays and human-computer interaction (HCI). However, most current eye trackers focus on 2D point-of-gaze (PoG) estimation and cannot provide accurate gaze depth. Concerning future applications such as HCI with 3D displays, we propose an integrated binocular eye-tracking device with stereo stimuli to provide highly accurate 3D PoG estimation. In our device, the 3D stereo imaging system can provide users with a friendly and immersive 3D visual experience without wearing any accessories. The eye-capturing system can directly record the users’ eye movements under 3D stimuli without disturbance, which can realize front views of eye images while ensuring the user’s wide field of view (FOV). A regression-based 3D eye-tracking model is built based on collected eye movement data under stereo stimuli. Our model estimates users’ 2D gaze with features defined by eye region landmarks and further estimates 3D gaze with a multi-source feature set constructed by comprehensive eye movement features and disparity features from stereo stimuli. Two test stereo scenes with different depths of field are designed to verify the model’s effectiveness. Experimental results of four-fold cross-validation show that the average error for 2D gaze estimation was 0.65°. For 3D gaze estimation, the average errors are 1.69 cm over the workspace volume 50 × 30 × 75 cm ³ and 0.14 m over the workspace volume 2.4 × 4.0 × 7.9 m ³ . Two preliminary validation experiments also demonstrate that our system has broad application prospects, such as 3D vision technologies and medical applications related to neurological diseases.

Eye-tracking technologies have been widely used in applications like psychological studies and human computer interactions (HCI). However, most current eye trackers focus on 2D point of gaze (PoG) estimation and cannot provide accurate gaze depth.Concerning future applications such as HCI with 3D displays, we propose a novel binocular eye tracking device with stereo stimuli to provide highly accurate 3D PoG estimation. In our device, the 3D stereo imaging system can provide users with a friendly and immersive 3D visual experience without wearing any accessories. The eye capturing system can directly record the users eye movements under 3D stimuli without disturbance. A regression based 3D eye tracking model is built based on collected eye movement data under stereo stimuli. Our model estimates users 2D gaze with features defined by eye region landmarks and further estimates 3D PoG with a multi source feature set constructed by comprehensive eye movement features and disparity features from stereo stimuli. Two test stereo scenes with different depths of field are designed to verify the model effectiveness. Experimental results show that the average error for 2D gaze estimation was 0.66\degree and for 3D PoG estimation, the average errors are 1.85~cm/0.15~m over the workspace volume 50~cm $\times$ 30~cm $\times$ 75~cm/2.4~m $\times$ 4.0~m $\times$ 7.9~m separately.