Collecting emotional physiological signals is significant in building affective Human-Computer Interactions (HCI). However, how to evoke subjects’ emotions efficiently in EEG-related emotional experiments is still a challenge. In this work, we developed a novel experimental paradigm that allows odors dynamically participate in different stages of video-evoked emotions, to investigate the efficiency of olfactory-enhanced videos in inducing subjects’ emotions; According to the period that the odors participated in, the stimuli were divided into four patterns, i.e., the olfactory-enhanced video in early/later stimulus periods (OVEP/OVLP), and the traditional videos in early/later stimulus periods (TVEP/TVLP). The differential entropy (DE) feature and four classifiers were employed to test the efficiency of emotion recognition. The best average accuracies of the OVEP, OVLP, TVEP, and TVLP were 50.54%, 51.49%, 40.22%, and 57.55%, respectively. The experimental results indicated that the OVEP significantly outperformed the TVEP on classification performance, while there was no significant difference between the OVLP and TVLP. Besides, olfactory-enhanced videos achieved higher efficiency in evoking negative emotions than traditional videos. Moreover, we found that the neural patterns in response to emotions under different stimulus methods were stable, and for Fp1, FP2, and F7, there existed significant differences in whether adopt the odors.

Conventional vibration sensing systems, equipped with specific sensors (e.g., accelerometer) and communication modules, are either expensive or cumbersome to deploy. Recently research community revisits this classic topic by taking advantage of off-the-shelf RFIDs. However, limited by low reading rate and long wavelength, current RFID based solutions can only sense low-frequency (e.g., below 100 Hz) mechanical vibrations with larger amplitude (e.g., $>\!\!5$ mm). To address the issue, this work presents TagSound, an RFID-based vibration sensing system that explores a tag's harmonic backscattering to recover high-frequency and tiny mechanical vibrations accurately. The key innovations are in two aspects: harmonics based sensing and a new recovery scheme . We implement TagSound with USRP platforms. Our comprehensive evaluation shows (i) TagSound can achieve a mean error of 0.37 Hz when detecting vibrations at frequencies below 100 Hz, and a mean error of 4.2 Hz even when the vibration frequency is up to 2500 Hz. (ii) TagSound can achieve a Hz-level frequency estimation even when the vibration amplitude is only 2 mm.

The paper proposes a new HCI mechanism for device-free gesture recognition on the table using acoustic signal, which can extend the gesture input and interactions beyond the tiny screen of mobile device and allow users to provide input without blocking screen view. Previous researches have either relied on additional devices (e.g., special wearable device and mouse) or required active acoustic signals which demand additional cost and be less prone to popularize, while we explore the device-free gesture recognition using passive acoustic signals. This technology is more challenging due to the lack of an effective approach to eliminate the inherent ambient noise disturbances and extract stable gesture features. We fuse both short time energy (STE) and zero-crossing rate (ZCR) to identify the effective signals from the original input, and leverage the Mel frequency cepstral coefficients (MFCC), cochlear filter cepstral coefficients (CFCC) to extract the stable features from different gestures. The unique features in support vector machine (SVM) classifier achieve a high gesture recognition accuracy from the noisy scenarios and mismatched conditions. Implementation on the Android system has realized real-time processing of the feature extraction and gesture recognition. Extensive evaluations show our algorithm has a better noise tolerant performance and the system could recognize seven common gestures (click, flip left/right, scroll up/down, zoom in/out) on smart devices with an accuracy of 93.2%.

Emotion recognition plays a vital role in building a harmonious society and emotional interaction. Recent research has demonstrated that multi-modal inter-channel correlations and insufficient emotion elicitation plague deep learning-based emotion identification techniques. To cope with these problems, we propose a transformer model of multi-modal and channel attention fusion (MCAF-Transformer). First, we employ an olfactory video approach to evoke emotional expression more fully and acquire EEG and EOG signal data. Second, the model makes full use of multi-modal channel information, time-domain and spatial-domain information of EEG and EOG signals, captures the correlation of different channels using channel attention, and improves the accuracy of emotion recognition by focusing on the global dependence on the temporal order using the transformer. We conducted extensive experiments on the olfactory video sentiment dataset, and the experimental results were correct at 94.63%. The results show that olfactory videos evoke emotion more adequately than pure videos and that the MCAF-Transformer model significantly outperforms other emotion recognition methods.

The widely deployed WiFi APs and ZigBee devices make the coexistence for these two networks more pervasive than ever. Previous solutions focus on the contention resolution in time domain, but would lead to the underutilized frequency resources. However, for frequency domain schemes, to enable efficient coexistence for WiFi and ZigBee networks, the WiFi nodes or ZigBee nodes have to select the appropriate channel for transmission. Such scheme will need strict cooperations among these two independent systems, and could not work with the conventional legacy systems seamlessly. In this paper, we propose a frequency overlay approach named COFFEE (COexist wiFi For zigbEE networks), where subcarriers interfering with ZigBee transmissions are nullified by WiFi. According to our basic evaluations, due to the relatively robust feature in WiFi/OFDM design, nullifying small portion of subcarriers would affect the WiFi performance slightly, but could improve the capability for coexistence significantly. Thus by using COFFEE, WiFi and ZigBee nodes can transmit their packets concurrently without any coordination. Also, COFFEE and conventional legacy WiFi systems could work together seamlessly. Furthermore, we implement COFFEE with USRP software radio platform and evaluate the performance under real wireless network scenarios. The extensive evaluations show that compared to the coesistence schemes in time domain, COFFEE is able to increase the network throughput by more than 300%. Even when the ZigBee node never knows the transmission time of WiFi packet, it still can provide concurrent lossless transmission by using COFFEE with only 10% to 15% WiFi throughput reduction.

Affective data is the basis of emotion recognition, which is mainly acquired through extrinsic elicitation. To investigate the enhancing effects of multi-sensory stimuli on emotion elicitation and emotion recognition, we designed an experimental paradigm involving visual, auditory, and olfactory senses. A multimodal emotional dataset (OVPD-II) that employed the video-only or video-odor patterns as the stimuli materials, and recorded the electroencephalogram (EEG) and electrooculogram (EOG) signals, was created. The feedback results reported by subjects after each trial demonstrated that the video-odor pattern outperformed the video-only pattern in evoking individuals’ emotions. To further validate the efficiency of the video-odor pattern, the transformer was employed to perform the emotion recognition task, where the highest accuracy reached 86.65% (66.12%) for EEG (EOG) modality with the video-odor pattern, which improved by 1.42% (3.43%) compared with the video-only pattern. What’s more, the hybrid fusion (HF) method combined with the transformer and joint training was developed to improve the performance of the emotion recognition task, which achieved classify accuracies of 89.50% and 88.47% for the video-odor and video-only patterns, respectively.