Background: Monitoring the body temperature of premature infants is vital, as it allows optimal temperature control and may provide early warning signs for severe diseases such as sepsis. Thermography may be a non-contact and wireless alternative to state-of-the-art, cable-based methods. For monitoring use in clinical practice, automatic segmentation of the different body regions is necessary due to the movement of the infant.Methods: This work presents and evaluates algorithms for automatic segmentation of infant body parts using deep learning methods. Based on a U-Net architecture, three neural networks were developed and compared. While the first two only used one imaging modality (visible light or thermography), the third applied a feature fusion of both. For training and evaluation, a dataset containing 600 visible light and 600 thermography images from 20 recordings of infants was created and manually labeled. In addition, we used transfer learning on publicly available datasets of adults in combination with data augmentation to improve the segmentation results.Results: Individual optimization of the three deep learning models revealed that transfer learning and data augmentation improved segmentation regardless of the imaging modality. The fusion model achieved the best results during the final evaluation with a mean Intersection-over-Union (mIoU) of 0.85, closely followed by the RGB model. Only the thermography model achieved a lower accuracy (mIoU of 0.75). The results of the individual classes showed that all body parts were well-segmented, only the accuracy on the torso is inferior since the models struggle when only small areas of the skin are visible.Conclusion: The presented multi-modal neural networks represent a new approach to the problem of infant body segmentation with limited available data. Robust results were obtained by applying feature fusion, cross-modality transfer learning and classical augmentation strategies.

Neonatal intensive care units provide vital medical support for premature infants. The key aspect in neonatal care is the continuous monitoring of vital signs measured using adhesive skin sensors. Since sensors can cause irritation of the skin and lead to infections, research focuses on contact-free, camera-based methods such as infrared thermography and photoplethysmography imaging. The development of image processing algorithms requires large datasets, but recording the necessary data from studies brings tremendous effort and costs. Therefore, realistic patient phantoms would be feasible to create a comprehensive dataset and validate image-based algorithms. This work describes the realization of a neonatal phantom which can simulate physiological vital parameters such as pulse rate and thermoregulation. It mimics the outer appearance of premature infants using a 3D printed base structure coated with several layers of modified, skin-colored silicone. A distribution of red and infrared LEDs in the scaffold enables the simulation of a PPG signal by mimicking pulsative light intensity changes on the skin. Additionally, the body temperature of the phantom is individually adjustable in several regions using heating elements. In the validation process for PPG simulation, the feasibility of setting different pulse frequencies and the variation of oxygen saturation levels was obtained. Furthermore, heating tests showed region-dependent temperature variations between 0.19C and 0.81C around the setpoint. In conclusion, the proposed neonatal phantom can be used to simulate a variety of vital parameters of preterm infants and, therefore, enables the implementation of image processing algorithms for the analysis of the medical state.

Premature infants are among the most vulnerable patients in a hospital. Due to numerous complications associated with immaturity, a continuous monitoring of vital signs with a high sensitivity and accuracy is required. Today, wired sensors are attached to the patient’s skin. However, adhesive electrodes can be potentially harmful as they can damage the very thin immature skin. Although unobtrusive monitoring systems using cameras show the potential to replace cable-based techniques, advanced image processing algorithms are data-driven and, therefore, need much data to be trained. Due to the low availability of public neonatal image data, a patient phantom could help to implement algorithms for the robust extraction of vital signs from video recordings. In this work, a camera-based system is presented and validated using a neonatal phantom, which enabled a simulation of common neonatal pathologies such as hypo-/hyperthermia and brady-/tachycardia. The implemented algorithm was able to continuously measure and analyze the heart rate via photoplethysmography imaging with a mean absolute error of 0.91 bpm, as well as the distribution of a neonate’s skin temperature with a mean absolute error of less than 0.55 °C. For accurate measurements, a temperature gain offset correction on the registered image from two infrared thermography cameras was performed. A deep learning-based keypoint detector was applied for temperature mapping and guidance for the feature extraction. The presented setup successfully detected several levels of hypo- and hyperthermia, an increased central-peripheral temperature difference, tachycardia and bradycardia.

Accurate and reliable skin temperature monitoring is critical for the thermoregulation of premature infants, but current methods using wired sensors are in