Driver distraction diverting drivers’ attention to unrelated tasks and decreasing the ability to control vehicles, has aroused widespread concern about driving safety. Previous studies have found that driving performance decreases after distraction and have used vehicle behavioral features to detect distraction. But how brain activity changes while distraction remains unknown. Electroencephalography (EEG), a reliable indicator of brain activities has been widely employed in many fields. However, challenges still exist in mining the distraction information of EEG in realistic driving scenarios with uncertain information. In this paper, we propose a novel framework based on Multi-scale entropy (MSE) in a sliding window and Bidirectional Long Short-term Memory Network (BiLSTM) to explore the distraction information of EEG to detect driver distraction based on multi-modality signals in real traffic. Firstly, MSE with sliding window is implemented to extract the EEG features to determine the distraction position. Statistical analysis of vehicle behavioral data is then performed to validate driving performance indeed changes around distraction position. Finally, we use BiLSTM to detect driver distraction with MSE and other traditional features. Our results show that MSE notably decreases after distraction. Consistent with the result of MSE, driving performance significantly deviates from the normal state after distraction. Besides, BiLSTM performance of MSE outperforms other entropy-based methods and is better than behavioral features. Additionally, the accuracy is improved again after adding MSE feature to behavioral features with a 3% increasement. The proposed framework is useful for mining brain activity information and driver distraction detection applications in realistic driving scenarios.
Emotion recognition based on electroencephalography (EEG) has attracted high interest in fields such as health care, user experience evaluation, and human–computer interaction (HCI), as it plays an important role in human daily life. Although various approaches have been proposed to detect emotion states in previous studies, there is still a need to further study the dynamic changes of EEG in different emotions to detect emotion states accurately. Entropy-based features have been proved to be effective in mining the complexity information in EEG in many areas. However, different entropy features vary in revealing the implicit information of EEG. To improve system reliability, in this paper, we propose a framework for EEG-based cross-subject emotion recognition using fused entropy features and a Bidirectional Long Short-term Memory (BiLSTM) network. Features including approximate entropy (AE), fuzzy entropy (FE), Rényi entropy (RE), differential entropy (DE), and multi-scale entropy (MSE) are first calculated to study dynamic emotional information. Then, we train a BiLSTM classifier with the inputs of entropy features to identify different emotions. Our results show that MSE of EEG is more efficient than other single-entropy features in recognizing emotions. The performance of BiLSTM is further improved with an accuracy of 70.05% using fused entropy features compared with that of single-type feature.
25 June 2024 in IEEE Transactions on Intelligent Transportation Systems
Driver distraction, a major cause of traffic crashes, is reported to reduce driving performance and be detected with vehicle behavioral features. It also induces physiological responses. Time and frequency-domain features of physiological signals have been used to study distraction, but they are susceptible to residual noise and tend to overlook complexity. Moreover, the resampling problem arises while analyzing physiological signals at multiple time scales. This paper proposes a novel framework based on multiscale entropy on absolute time scales (MSaE) and bidirectional long short-term memory (BiLSTM) network to mine the distraction information in multi-modality physiological signals and detect distraction automatically. Firstly, an entropy-based resampling method is adopted to find the suitable downsampling rates of electroencephalography (EEG), electrocardiogram (ECG), and electromyography (EMG). Then, calculating entropy with absolute time scales instead of relative time scales in a sliding window is utilized to explore the fluctuations of each signal while distraction. Afterward, ReliefF is selected from conventional feature selectors to identify the optimal feature set for each signal. Finally, BiLSTM with time dependency is designed to detect driver distraction with the selected feature set. The results illustrate significant distinctions in the MSaE of multiple physiological signals between normal and distracted driving. Additionally, MSaE, superior to traditional features, is selected as the most discriminative feature for each signal in distraction mining. Furthermore, the accuracy is further improved by about 8%, incorporating multi-modality features rather than vehicle behavioral features. This study indicates the potential of employing various signals to understand and detect driver distraction effectively.