We present SpeakingFaces as a publicly-available large-scale multimodal dataset developed to support machine learning research in contexts that utilize a combination of thermal, visual, and audio data streams; examples include human–computer interaction, biometric authentication, recognition systems, domain transfer, and speech recognition. SpeakingFaces is comprised of aligned high-resolution thermal and visual spectra image streams of fully-framed faces synchronized with audio recordings of each subject speaking approximately 100 imperative phrases. Data were collected from 142 subjects, yielding over 13,000 instances of synchronized data (∼3.8 TB). For technical validation, we demonstrate two baseline examples. The first baseline shows classification by gender, utilizing different combinations of the three data streams in both clean and noisy environments. The second example consists of thermal-to-visual facial image translation, as an instance of domain transfer.

In this work, we present an open-source stochastic epidemic simulator calibrated with extant epidemic experience of COVID-19. The simulator models a country as a network representing each node as an administrative region. The transportation connections between the nodes are modeled as the edges of this network. Each node runs a Susceptible-Exposed-Infected-Recovered (SEIR) model and population transfer between the nodes is considered using the transportation networks which allows modeling of the geographic spread of the disease. The simulator incorporates information ranging from population demographics and mobility data to health care resource capacity, by region, with interactive controls of system variables to allow dynamic and interactive modeling of events. The single-node simulator was validated using the thoroughly reported data from Lombardy, Italy. Then, the epidemic situation in Kazakhstan as of 31 May 2020 was accurately recreated. Afterward, we simulated a number of scenarios for Kazakhstan with different sets of policies. We also demonstrate the effects of region-based policies such as transportation limitations between administrative units and the application of different policies for different regions based on the epidemic intensity and geographic location. The results show that the simulator can be used to estimate outcomes of policy options to inform deliberations on governmental interdiction policies.

Face detection and subsequent localization of facial landmarks are the primary steps in many face applications. Numerous algorithms and benchmark datasets have been introduced to develop robust models for the visible domain. However, varying conditions of illumination still pose challenging problems. In this regard, thermal cameras are employed to address this problem, because they operate on longer wavelengths. However, thermal face and facial landmark detection in the wild is an open research problem because most of the existing thermal datasets were collected in controlled environments. In addition, many of them were not annotated with face bounding boxes and facial landmarks. In this work, we present a thermal face dataset with manually labeled bounding boxes and facial landmarks to address these problems. The dataset contains 9,982 images of 147 subjects collected under controlled and uncontrolled conditions. As a baseline, we trained the YOLOv5 object detection model and its adaptation for face detection, YOLO5Face, on our dataset. In addition to our test set, we evaluated the models on the external RWTH-Aachen thermal face dataset to show the efficacy of our dataset. We have made the dataset, source code, and pre-trained models publicly available at https://github.com/IS2AI/TFW to bolster research in thermal face analysis.

In the contemporary world, inspection operations have become a critical component of infrastructure maintenance. Over the years, the demand for comprehensive inspection of pipes, both internally and externally, has grown increasingly complex and challenging. Consequently, there is a pressing need for significant advancements in in-pipe robots, particularly in the areas of inspection speed, defect detection precision, and overall reliability. Recent developments in new devices and sensors have markedly improved our capability to inspect and diagnose defects within pipes with greater accuracy. Furthermore, the application of machine learning tools has optimized the inspection process, enhancing the detection and recognition of potential pipe defects, such as rust, blockages, and welding anomalies. This research introduces a novel mobile robot platform specifically designed for pipe inspection. It integrates an advanced machine learning model that effectively detects and identifies key pipe defects, including rust, compromised welding quality, and pipe deformation. Additionally, this platform offers enhancements in inspection speed. The integration of these technologies represents a significant stride in the field of infrastructure maintenance, setting a new standard for efficiency and precision in pipe inspection.

Goal: The COVID-19 pandemic has emerged as the most severe public health crisis in over a century. As of January 2021, there are more than 100 million cases and 2.1 million deaths. For informed decision making, reliable statistical data and capable simulation tools are needed. Our goal is to develop an epidemic simulator that can model the effects of random population testing and contact tracing. Methods: Our simulator models individuals as particles with the position, velocity, and epidemic status states on a 2D map and runs an SEIR epidemic model with contact tracing and testing modules. The simulator is available on GitHub under the MIT license. Results: The results show that the synergistic use of contact tracing and massive testing is effective in suppressing the epidemic (the number of deaths was reduced by 72%). Conclusions: The Particle-based COVID-19 simulator enables the modeling of intervention measures, random testing, and contact tracing, for epidemic mitigation and suppression.

Human pose estimation has many applications in action recognition, human-robot interaction, motion capture, augmented reality, sports analytics, and healthcare. Numerous datasets and deep learning models have been developed for human pose estimation within the visible domain. However, poor lighting conditions and privacy issues persist. These challenges can be addressed using thermal cameras; however, there is a limited number of annotated thermal human pose datasets for training deep learning models. Previously, we presented the OpenThermalPose dataset with 6,090 thermal images of 31 subjects and 14,315 annotated human instances. In this work, we extend OpenThermalPose with more thermal images, human instances, and poses. The extended dataset, OpenThermalPose2, contains 21,125 elaborately annotated human instances within 11,391 thermal images of 170 subjects. To show the efficacy of OpenThermalPose2, we trained the YOLOv8-pose and YOLO11-pose models on the dataset. The experimental results showed that models trained with OpenThermalPose2 outperformed the previous YOLOv8-pose models trained with OpenThermalPose. Additionally, we optimized the YOLO11-pose models trained on OpenThermalPose2 by converting their checkpoints from PyTorch to TensorRT formats. We deployed the PyTorch and TensorRT models on an NVIDIA Jetson AGX Orin 64GB and measured their inference time and accuracy. The TensorRT models using half-precision floating-point (FP16) achieved the best balance between speed and accuracy, making them suitable for real-time applications. We have made the dataset, source code, and pre-trained models publicly available at https://github.com/IS2AI/OpenThermalPose to bolster research in this field.