Adapting intelligent context-aware systems (CAS) to future operating rooms (OR) aims to improve situational awareness and provide surgical decision support systems to medical teams. CAS analyzes data streams from available devices during surgery and communicates real-time knowledge to clinicians. Indeed, recent advances in computer vision and machine learning, particularly deep learning, paved the way for extensive research to develop CAS. In this work, a deep learning approach for analyzing laparoscopic videos for surgical phase recognition, tool classification, and weakly-supervised tool localization in laparoscopic videos was proposed. The ResNet-50 convolutional neural network (CNN) architecture was adapted by adding attention modules and fusing features from multiple stages to generate better-focused, generalized, and well-representative features. Then, a multi-map convolutional layer followed by tool-wise and spatial pooling operations was utilized to perform tool localization and generate tool presence confidences. Finally, the long short-term memory (LSTM) network was employed to model temporal information and perform tool classification and phase recognition. The proposed approach was evaluated on the Cholec80 dataset. The experimental results (i.e., 88.5% and 89.0% mean precision and recall for phase recognition, respectively, 95.6% mean average precision for tool presence detection, and a 70.1% F1-score for tool localization) demonstrated the ability of the model to learn discriminative features for all tasks. The performances revealed the importance of integrating attention modules and multi-stage feature fusion for more robust and precise detection of surgical phases and tools.
Measurement of accurate tidal volumes based on respiration-induced surface movements of the upper body would be valuable in clinical and sports monitoring applications, but most current methods lack the precision, ease of use, or cost effectiveness required for wide-scale uptake. In this paper, the theoretical ability of different sensors, such as inertial measurement units, strain gauges, or circumference measurement devices to determine tidal volumes were investigated, scrutinised and evaluated. Sixteen subjects performed different breathing patterns of different tidal volumes, while using a motion capture system to record surface motions and a spirometer as a reference to obtain tidal volumes. Subsequently, the motion-capture data were used to determine upper-body circumferences, tilt angles, distance changes, movements and accelerations—such data could potentially be measured using optical encoders, inertial measurement units, or strain gauges. From these parameters, the measurement range and correlation with the volume signal of the spirometer were determined. The highest correlations were found between the spirometer volume and upper body circumferences; surface deflection was also well correlated, while accelerations carried minor respiratory information. The ranges of thorax motion parameters measurable with common sensors and the values and correlations to respiratory volume are presented. This article thus provides a novel tool for sensor selection for a smart shirt analysis of respiration.
The measurement of respiratory volume based on upper body movements by means of a smart shirt is increasingly requested in medical applications. This research used upper body surface motions obtained by a motion capture system, and two regression methods to determine the optimal selection and placement of sensors on a smart shirt to recover respiratory parameters from benchmark spirometry values. The results of the two regression methods (Ridge regression and the least absolute shrinkage and selection operator (Lasso)) were compared. This work shows that the Lasso method offers advantages compared to the Ridge regression, as it provides sparse solutions and is more robust to outliers. However, both methods can be used in this application since they lead to a similar sensor subset with lower computational demand (from exponential effort for full exhaustive search down to the order of O (n2)). A smart shirt for respiratory volume estimation could replace spirometry in some cases and would allow for a more convenient measurement of respiratory parameters in home care or hospital settings.