Adapting intelligent context-aware systems (CAS) to future operating rooms (OR) aims to improve situational awareness and provide surgical decision support systems to medical teams. CAS analyzes data streams from available devices during surgery and communicates real-time knowledge to clinicians. Indeed, recent advances in computer vision and machine learning, particularly deep learning, paved the way for extensive research to develop CAS. In this work, a deep learning approach for analyzing laparoscopic videos for surgical phase recognition, tool classification, and weakly-supervised tool localization in laparoscopic videos was proposed. The ResNet-50 convolutional neural network (CNN) architecture was adapted by adding attention modules and fusing features from multiple stages to generate better-focused, generalized, and well-representative features. Then, a multi-map convolutional layer followed by tool-wise and spatial pooling operations was utilized to perform tool localization and generate tool presence confidences. Finally, the long short-term memory (LSTM) network was employed to model temporal information and perform tool classification and phase recognition. The proposed approach was evaluated on the Cholec80 dataset. The experimental results (i.e., 88.5% and 89.0% mean precision and recall for phase recognition, respectively, 95.6% mean average precision for tool presence detection, and a 70.1% F1-score for tool localization) demonstrated the ability of the model to learn discriminative features for all tasks. The performances revealed the importance of integrating attention modules and multi-stage feature fusion for more robust and precise detection of surgical phases and tools.
Measurement of accurate tidal volumes based on respiration-induced surface movements of the upper body would be valuable in clinical and sports monitoring applications, but most current methods lack the precision, ease of use, or cost effectiveness required for wide-scale uptake. In this paper, the theoretical ability of different sensors, such as inertial measurement units, strain gauges, or circumference measurement devices to determine tidal volumes were investigated, scrutinised and evaluated. Sixteen subjects performed different breathing patterns of different tidal volumes, while using a motion capture system to record surface motions and a spirometer as a reference to obtain tidal volumes. Subsequently, the motion-capture data were used to determine upper-body circumferences, tilt angles, distance changes, movements and accelerations—such data could potentially be measured using optical encoders, inertial measurement units, or strain gauges. From these parameters, the measurement range and correlation with the volume signal of the spirometer were determined. The highest correlations were found between the spirometer volume and upper body circumferences; surface deflection was also well correlated, while accelerations carried minor respiratory information. The ranges of thorax motion parameters measurable with common sensors and the values and correlations to respiratory volume are presented. This article thus provides a novel tool for sensor selection for a smart shirt analysis of respiration.
The measurement of respiratory volume based on upper body movements by means of a smart shirt is increasingly requested in medical applications. This research used upper body surface motions obtained by a motion capture system, and two regression methods to determine the optimal selection and placement of sensors on a smart shirt to recover respiratory parameters from benchmark spirometry values. The results of the two regression methods (Ridge regression and the least absolute shrinkage and selection operator (Lasso)) were compared. This work shows that the Lasso method offers advantages compared to the Ridge regression, as it provides sparse solutions and is more robust to outliers. However, both methods can be used in this application since they lead to a similar sensor subset with lower computational demand (from exponential effort for full exhaustive search down to the order of O (n2)). A smart shirt for respiratory volume estimation could replace spirometry in some cases and would allow for a more convenient measurement of respiratory parameters in home care or hospital settings.
Deep learning approaches have been explored for surgical tool classification in laparoscopic videos. Convolutional neural networks (CNN) are prominent among the proposed approaches. However, concerns about the robustness and generalisability of CNN approaches have been raised. This paper evaluates CNN generalisability across different procedures and in data from different surgical settings. Moreover, generalisation performance to new types of procedures is assessed and insights are provided into the effect of increasing the size and representativeness of training data on the generalisation capabilities of CNN. Five experiments were conducted using three datasets. The DenseNet-121 model showed high generalisation capability within the dataset, with a mean average precision of 93%. However, the model performance diminished on data from different surgical sites and across procedure types (27% and 38%, respectively). The generalisation performance of the CNN model was improved by increasing the quantity of training videos on data of the same procedure type (the best improvement was 27%). These results highlight the importance of evaluating the performance of CNN models on data from unseen sources in order to determine their real classification capabilities. While the analysed CNN model yielded reasonably robust performance on data from different subjects, it showed a moderate reduction in performance for different surgical settings.
Minimal invasive surgery, more specifically laparoscopic surgery, is an active topic in the field of research. The collaboration between surgeons and new technologies aims to improve operation procedures as well as to ensure the safety of patients. An integral part of operating rooms modernization is the real-time communication between the surgeon and the data gathered using the numerous devices during surgery. A fundamental tool that can aid surgeons during laparoscopic surgery is the recognition of the different phases during an operation. Current research has shown a correlation between the surgical tools utilized and the present phase of surgery. To this end, a robust surgical tool classifier is desired for optimal performance. In this paper, a deep learning framework embedded with a custom attention module, the P-CSEM, has been proposed to refine the spatial features for surgical tool classification in laparoscopic surgery videos. This approach utilizes convolutional neural networks (CNNs) integrated with P-CSEM attention modules at different levels of the architecture for improved feature refinement. The model was trained and tested on the popular, publicly available Cholec80 database. Results showed that the attention integrated model achieved a mean average precision of 93.14%, and visualizations revealed the ability of the model to adhere more towards features of tool relevance. The proposed approach displays the benefits of integrating attention modules into surgical tool classification models for a more robust and precise detection.