Non-contact visual monitoring of vital signs in neonatology has been demonstrated by several recent studies in ideal scenarios where the baby is calm and there is no medical or parental intervention. Similar to contact monitoring methods (e.g., ECG, pulse oximeter) the camera-based solutions suffer from motion artifacts. Therefore, during care and the infants’ active periods, calculated values typically differ largely from the real ones. In this way, our main contribution to existing remote camera-based techniques is to detect and classify such situations with a high level of confidence. Our algorithms can not only evaluate quiet periods, but can also provide continuous monitoring. Altogether, our proposed algorithms can measure pulse rate, breathing rate, and to recognize situations such as medical intervention or very active subjects using only a single camera, while the system does not exceed the computational capabilities of average CPU-GPU-based hardware. The performance of the algorithms was evaluated on our database collected at the I^st Dept. of Neonatology of Pediatrics, Dept of Obstetrics and Gynecology, Semmelweis University, Budapest, Hungary.

The objective evaluation of an infant’s activity and sleep pattern is critical in improving the comfort of the babies and ensuring the proper amount of quality sleep. The predefined behavioral states of an infant describe their consciousness and arousal level. The different states are characterized by different movements, body tone, eye movements and breath patterns. To recognize and adapt to these states is an essential part of development-friendly caring. It affects the neonate’s sleep, influencing their brain development, while improving the bonding between mother and baby, and feeding is more successful during the state of quiet awakened. It can be a more difficult task to determine the level of arousal in premature neonates. In preterm clinics, the general practice is continuous observation, requiring the attention of the hospital staff. To create an automated, more objective system, helping the hospital staff and the parents, we developed a multi-RNN (multi-recurrent neural network) network-based solution to solve this classification problem, which works on a time-series-like feature set, extracted from cameras’ video feeds. The set is composed of video actigraphy features, video-based respiration signal and additional descriptors. We separate infant caring from undisturbed presence based on our previous ensemble network solution. The network was trained and evaluated using our database of 402 h of footage, collected at the Neonatal Intensive Care Unit, Dept. of Neonatology of Pediatrics, Dept. of Obstetrics and Gynecology, Semmelweis University, Budapest, Hungary, with all-day recordings of 10 babies.

We introduce a novel system for automatic assessment of newborn and preterm infant behavior—including activity levels, behavioral states, and sleep–wake cycles—in clinical settings for streamlining care and minimizing healthcare professionals’ workload. While vital signs are routinely monitored, the previously mentioned assessments require labor-intensive direct observation. Research so far has already introduced non- and minimally invasive solutions. However, we developed a system that automatizes the preceding evaluations in a non-contact way using deep learning algorithms. In this work, we provide a Gated Recurrent Unit (GRU)-stack-based solution that works on a dynamic feature set generated by computer vision methods from the cameras’ video feed and patient monitor to classify the activity phases of infants adapted from the NIDCAP (Newborn Individualized Developmental Care Program) scale. We also show how pulse rate variability (PRV) data could improve the performance of the classification. The network was trained and evaluated on our own database of 108 h collected at the Neonatal Intensive Care Unit, Dept. of Neonatology of Pediatrics, Semmelweis University, Budapest, Hungary.