27 March 2019 in ACM Transactions on Interactive Intelligent Systems
Recent research has shown that reliable recognition of sign language words and phrases using user-friendly and noninvasive armbands is feasible and desirable. This work provides an analysis and implementation of including fingerspelling recognition (FR) in such systems, which is a much harder problem due to lack of distinctive hand movements. A novel algorithm called DyFAV (Dynamic Feature Selection and Voting) is proposed for this purpose that exploits the fact that fingerspelling has a finite corpus (26 alphabets for the American Sign Language (ASL)). Detailed analysis of the algorithm used as well as comparisons with other traditional machine-learning algorithms is provided. The system uses an independent multiple-agent voting approach to identify letters with high accuracy. The independent voting of the agents ensures that the algorithm is highly parallelizable and thus recognition times can be kept low to suit real-time mobile applications. A thorough explanation and analysis is presented on results obtained on the ASL alphabet corpus for nine people with limited training. An average recognition accuracy of 95.36% is reported and compared with recognition results from other machine-learning techniques. This result is extended by including six additional validation users with data collected under similar settings as the previous dataset. Furthermore, a feature selection schema using a subset of the sensors is proposed and the results are evaluated. The mobile, noninvasive, and real-time nature of the technology is demonstrated by evaluating performance on various types of Android phones and remote server configurations. A brief discussion of the user interface is provided along with guidelines for best practices.
4 January 2023 in Frontiers in Neuroimaging
Objective: Accurate localization of a seizure onset zone (SOZ) from independent components (IC) of resting-state functional magnetic resonance imaging (rs-fMRI) improves surgical outcomes in children with drug-resistant epilepsy (DRE). Automated IC sorting has limited success in identifying SOZ localizing ICs in adult normal rs-fMRI or uncategorized epilepsy. Children face unique challenges due to the developing brain and its associated surgical risks. This study proposes a novel SOZ localization algorithm (EPIK) for children with DRE.Methods: EPIK is developed in a phased approach, where fMRI noise-related biomarkers are used through high-fidelity image processing techniques to eliminate noise ICs. Then, the SOZ markers are used through a maximum likelihood-based classifier to determine SOZ localizing ICs. The performance of EPIK was evaluated on a unique pediatric DRE dataset (n= 52). A total of 24 children underwent surgical resection or ablation of an rs-fMRI identified SOZ, concurrently evaluated with an EEG and anatomical MRI. Two state-of-art techniques were used for comparison: (a) least squares support-vector machine and (b) convolutional neural networks. The performance was benchmarked against expert IC sorting and Engel outcomes for surgical SOZ resection or ablation. The analysis was stratified across age and sex.Results: EPIK outperformed state-of-art techniques for SOZ localizing IC identification with a mean accuracy of 84.7% (4% higher), a precision of 74.1% (22% higher), a specificity of 81.9% (3.2% higher), and a sensitivity of 88.6% (16.5% higher). EPIK showed consistent performance across age and sex with the best performance in those < 5 years of age. It helped achieve a ~5-fold reduction in the number of ICs to be potentially analyzed during pre-surgical screening.Significance: Automated SOZ localization from rs-fMRI, validated against surgical outcomes, indicates the potential for clinical feasibility. It eliminates the need for expert sorting, outperforms prior automated methods, and is consistent across age and sex.
3 May 2024 in IEEE Transactions on Artificial Intelligence
Supervised artificial intelligence (AI) techniques are good at learning class specific characteristic properties despite variance across samples. However, for rare class classification, challenges arise due to class imbalance. To the contrary, knowledge based techniques encode class specific information without class labels but face difficulties in parsing knowledge which is often vague, uncertain, and result in high intra-class variability. This manuscript presents a human-AI collaboration methodology to integrate AI with expert knowledge for rare class classification, mitigating class imbalance and intra-class variability effects. We present a formal framework for expert knowledge representation using logical connectives of atomic propositions, a rule refinement strategy to derive class specific machine checkable formulas, and a rule implementation strategy that extracts explainable partitions of rare class expert rules for its recognition. A knowledge-AI integration strategy is presented that uses entropy imbalance gain and Gini index to quantify class imbalance and intra-class variability, and orchestrates supervised AI and expert knowledge machines to effectively identify rare class through human-AI collaboration with reduced human effort. We apply proposed integration framework to develop DeepXSOZ that identifies seizure onset zones (SOZ) in focal epilepsy patients from resting state functional magnetic resonance imaging. DeepXSOZ’s performance is validated on multi-center datasets against anatomical MRI based manual SOZ identification, and Engel outcomes after surgical SOZ alteration. This human-AI collaboration demonstrates increased F1 score compared to state-of-the-art “AI-only” techniques, minimal data leakage effect with statistically similar performance across multicenter datasets without fine tuning, consistent results across age and gender, and reduced manual effort.
11 January 2024 in Frontiers in Neurology
We evaluated whether integration of expert guidance on seizure onset zone (SOZ) identification from resting state functional MRI (rs-fMRI) connectomics combined with deep learning (DL) techniques enhances the SOZ delineation in patients with refractory epilepsy (RE), compared to utilizing DL alone. Rs-fMRI was collected from 52 children with RE who had subsequently undergone ic-EEG and then, if indicated, surgery for seizure control (n = 25). The resting state functional connectomics data were previously independently classified by two expert epileptologists, as indicative of measurement noise, typical resting state network connectivity, or SOZ. An expert knowledge integrated deep network was trained on functional connectomics data to identify SOZ. Expert knowledge integrated with DL showed a SOZ localization accuracy of 84.8 ± 4.5% and F1 score, harmonic mean of positive predictive value and sensitivity, of 91.7 ± 2.6%. Conversely, a DL only model yielded an accuracy of <50% (F1 score 63%). Activations that initiate in gray matter, extend through white matter, and end in vascular regions are seen as the most discriminative expert-identified SOZ characteristics. Integration of expert knowledge of functional connectomics can not only enhance the performance of DL in localizing SOZ in RE but also lead toward potentially useful explanations of prevalent co-activation patterns in SOZ. RE with surgical outcomes and preoperative rs-fMRI studies can yield expert knowledge most salient for SOZ identification.
17 July 2020 in IEEE Transactions on Industrial Informatics
Safety regulation of safety-critical agent-based Cyber-physical systems which are manufactured in large scale such as next-gen aircrafts, autonomous driving vehicles, and medical devices is multi-faceted problem. CPS deployments can be presented with new safety-critical scenarios and novel inputs. Hence, operational characteristics of the CPS can be quite different from its safety approved design. This paper considers a safety assurance solution where operational data from the sensors and actuators in the field of deployment is fed back to the manufacturing process through the Internet of Things infrastructure to assure and improve operational safety. It considers two cases: a) Model- Aware, where the safety assured CPS design is fully specified, and b) Model-Agnostic, where limited specifications exist. For both the cases, it presents a data science based approach, N-HyMn, that learns a hybrid automaton model of the operational characteristics of the CPS from the input/output traces of the observable parameters. For model-aware case, it investigates the presence of inconsistencies between the learned model and the specifications model provided by the manufacturer, thus facilitating the detection of safety problems that may have been overlooked. For the model-agnostic case, it can detect potential safety failures. We show the usage of N-HyMn on the Medtronic Minimed 670G system. NHyMN correctly infers the hybrid automaton specifications of the Minimed 670G and was able to detect a self-adaptation mechanism that is not declared explicitly in the certification documents of the U.S. Food and Drug Administration. NHyMN has a computational complexity of $O(kn^2)$ , where $k$ is the number of samples in the I/O trace, and n is the number of continuous variables.