This study explores an innovative approach to early Parkinson’s disease (PD) detection by analyzing speech data collected using a mixed reality (MR) system. A total of 57 Polish participants, including PD patients and healthy controls, performed five speech tasks while using an MR head-mounted display (HMD). Speech data were recorded and analyzed to extract acoustic and linguistic features, which were then evaluated using machine learning models, including logistic regression, support vector machines (SVMs), random forests, AdaBoost, and XGBoost. The XGBoost model achieved the best performance, with an F1-score of 0.90 ± 0.05 in the story-retelling task. Key features such as MFCCs (mel-frequency cepstral coefficients), spectral characteristics, RASTA-filtered auditory spectrum, and local shimmer were identified as significant in detecting PD-related speech alterations. Additionally, state-of-the-art deep learning models (wav2vec2, HuBERT, and WavLM) were fine-tuned for PD detection. HuBERT achieved the highest performance, with an F1-score of 0.94 ± 0.04 in the diadochokinetic task, demonstrating the potential of deep learning to capture complex speech patterns linked to neurodegenerative diseases. This study highlights the effectiveness of combining MR technology for speech data collection with advanced machine learning (ML) and deep learning (DL) techniques, offering a non-invasive and high-precision approach to PD diagnosis. The findings hold promise for broader clinical applications, advancing the diagnostic landscape for neurodegenerative disorders.
26 February 2025 in Biomedical Signal Processing and Control
This study introduces an innovative multimodal system designed to support the assessment of Parkinson’s disease, atypical Parkinsonian syndromes, and Huntington’s disease by integrating mixed-reality (MR) technology, specifically using augmented reality goggles. The system captures symptoms related to neurodegenerative disorders through detailed sensor data, including hand movements, gait, and gaze patterns. Central to our methodology is an interactive, game-like experience lasting 30 min, comprising 17 tasks that evaluate motor skills, speech, memory, cognitive functions, gait, and gaze. These tasks were developed in collaboration with neurologists and draw upon established clinical paradigms. This engaging experience, combined with traditional clinical tests, provides a comprehensive assessment of both physical and cognitive capabilities. Rather than aiming to provide a diagnostic decision, this study focuses on evaluating the feasibility and usability of the system in a clinical setting. By automating diagnostic tasks and utilizing MR technology, our approach enhances patient comfort and offers a more immersive and efficient assessment process. The system was evaluated with 15 neurodegenerative patients and 32 healthy subjects, demonstrating high scores in both immersion and usability across both groups. This work represents a first step toward integrating MR into clinical workflows for neurodegenerative disease assessment.