Yue Gu, Kangning Yang, Shiyu Fu, Shuhong Chen, Xinyu Li, Ivan Marsic. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018.
We present a system for activity recognition from passive RFID data using a deep convolutional neural network. We directly feed the RFID data into a deep convolutional neural network for activity recognition instead of selecting features and using a cascade structure that first detects object use from RFID data followed by predicting the activity. Because our system treats activity recognition as a multi-class classification problem, it is scalable for applications with large number of activity classes. We tested our system using RFID data collected in a trauma room, including 14 hours of RFID data from 16 actual trauma resuscitations. Our system outperformed existing systems developed for activity recognition and achieved similar performance with process-phase detection as systems that require wearable sensors or manually-generated input. We also analyzed the strengths and limitations of our current deep learning architecture for activity recognition from RFID data.
13 September 2018 in International Conference on Acoustics, Speech, and Signal Processing (ICASSP)
In this paper, we present a novel deep multimodal framework to predict human emotions based on sentence-level spoken language. Our architecture has two distinctive characteristics. First, it extracts the high-level features from both text and audio via a hybrid deep multimodal structure, which considers the spatial information from text, temporal information from audio, and high -level associations from low-level handcrafted features. Second, we fuse all features by using a three-layer deep neural network to learn the correlations across modalities and train the feature extraction and fusion modules together, allowing optimal global fine-tuning of the entire structure. We evaluated the proposed framework on the IEMOCAP dataset. Our result shows promising performance, achieving 60.4% in weighted accnrar-v for five emotion categories.
We describe a novel and practical activity recognition system for dynamic and complex medical settings using only passive RFID technology. Our activity recognition approach is based on the use of objects that are specific for a given activity. The object-use status is detected from RFID data and the activities are predicted from the statuses of use of different objects. We tagged 10 objects in a trauma room of an emergency department and recorded RFID data for 10 actual trauma resuscitations. More than 20,000 seconds of data were collected and used for analysis. The system achieved a 96% overall accuracy with a 0.74 F-score for detecting use of 10 common resuscitation objects and 95% accuracy with a 0.30 F Score for activity recognition of 10 medical activities.
1 September 2018 in Journal of Biomedical Informatics