Background: Regulation of temperature is clinically important in the care of neonates because it has a significant impact on prognosis. Although probes that make contact with the skin are widely used to monitor temperature and provide spot central and peripheral temperature information, they do not provide details of the temperature distribution around the body. Although it is possible to obtain detailed temperature distributions using multiple probes, this is not clinically practical. Thermographic techniques have been reported for measurement of temperature distribution in infants. However, as these methods require manual selection of the regions of interest (ROIs), they are not suitable for introduction into clinical settings in hospitals. Here, we describe a method for segmentation of thermal images that enables continuous quantitative contactless monitoring of the temperature distribution over the whole body of neonates. Methods: The semantic segmentation method, U-Net, was applied to thermal images of infants. The optimal combination of Weight Normalization, Group Normalization, and Flexible Rectified Linear Unit (FReLU) was evaluated. U-Net Generative Adversarial Network (U-Net GAN) was applied to thermal images, and a Self-Attention (SA) module was finally applied to U-Net GAN (U-Net GAN + SA) to improve precision. The semantic segmentation performance of these methods was evaluated. Results: The optimal semantic segmentation performance was obtained with application of FReLU and Group Normalization to U-Net, showing accuracy of 92.9% and Mean Intersection over Union (mIoU) of 64.5%. U-Net GAN improved the performance, yielding accuracy of 93.3% and mIoU of 66.9%, and U-Net GAN + SA showed further improvement with accuracy of 93.5% and mIoU of 70.4%. Conclusions: FReLU and Group Normalization are appropriate semantic segmentation methods for application to neonatal thermal images. U-Net GAN and U-Net GAN + SA significantly improved the mIoU of segmentation.
Thermal images are widely used for various healthcare applications and advanced research. However, thermal images captured by smartphone thermal cameras are not accurate for monitoring human body temperature due to the small body that is vulnerable to temperature change. In this paper, we propose ThermalWrist, a dynamic offset correction method for thermal images captured by smartphone thermal cameras. We fully utilize the characteristic that is specific to thermal cameras: the relative temperatures in a single thermal image are highly reliable, although the absolute temperatures fluctuate frequently. To correct the offset error, ThermalWrist combines thermal images with a reliable absolute temperature obtained by a wristband sensor based on the above characteristic. The evaluation results in an indoor air-conditioned environment shows that the mean absolute error and the standard deviation of face temperature measurement error decrease by 49.4% and 64.9%, respectively. In addition, Pearson’s correlation coefficient increases by 112%, highlighting the effectiveness of ThermalWrist. We also investigate the limitation with respect to the ambient temperature where ThermalWrist works effectively. The result shows ThermalWrist works well in the normal office environment, which is 22.91 °C and above.
Human Activity Recognition (HAR) has attracted considerable attention in recent years due to its potential applications in healthcare, smart homes, and security. Wi-Fi Channel State Information (CSI) is a promising sensor modality for HAR, providing a device-free and low-cost solution. However, building environment-independent models for HAR using Wi-Fi CSI remains a significant challenge. In this paper, we present a deep learning-based activity recognition system that exploits CSI measurements obtained from one or more environments to deliver consistent and accurate performance even in unseen environments. Our system employs a multi-task learning approach that is based on an encoder-decoder network architecture. This enables the encoder part of this architecture to mitigate the environment-dependent factors and extract a rich and environment-invariant representation. To evaluate the proposed system, we collected CSI samples for six activities pursued by three participants in four distinct environments. The results demonstrate the efficacy of the proposed system in achieving environment-independent HAR with an average accuracy of 80%. Additionally, the results validate the superiority of our method over environment-specific models by a minimum margin of 6% in cases of limited data.
The development of sensing technologies has enabled the estimation of human thermal sensation, which helps efficient heating, ventilation, and air conditioning control. Recently, machine learning techniques using physiological and environmental sensors have been proposed. However, in most existing literature, the personal thermal sensation is estimated using only opportunistic data as features, and evaluations are conducted in a static environment. In real situations, we are exposed to a dynamic environment, owing to human mobility and changes in airflow. Therefore, it is necessary to estimate personal thermal sensation in a dynamic surrounding environment. In this study, we propose TSVNet, a deep-learning-based method reflecting time-series changes to address the dynamic environment. We collected data for a total of 123 days, which included the dynamic environment data from 21 subjects, for the evaluation. The results indicate that our method improves F1-score by 5.8% compared with a baseline method. We also design a data balancing method for regression problems on imbalanced datasets, including time-series data. In addition, the result of the lookback time evaluation shows that the use of physiological information in the previous 10 minutes improves the performance of the method.
Behind many traffic accidents, there are more frequent minor incidents (risky traffic situations) that may lead to severe accidents. Analyzing such minor incidents effectively reduces accidents, but the challenge is to design a method to collect and analyze such incident information. In this paper, we propose a novel platform that aggregates behavioral data from pedestrians and drivers using their smartphones and recognizes risky traffic situations from the aggregated data. We design a two-stage approach where the smartphones of pedestrians and vehicles act as local anomaly detectors for triggering the event detector and classifier in the post-stage at the cloud server to suppress the processing and communication overhead. We also introduce an unsupervised learning system to cope with unseen risky situations enabled by joint utilization of the autoencoder-based anomaly detector and the risky situation classifier. The evaluation is conducted through both simulation and real experiments. The simulation result shows the risky situation detector achieves an F-measure of 0.89. We also collected real data at a car driving course to evaluate the risky situation classifier. From the results, we have confirmed that the proposed method succeeded in classifying three risky traffic situations involving pedestrians and/or vehicles with an accuracy of 89.3%.