Neonatal Endotracheal Intubation (ETI) is a critical resuscitation skill that requires tremendous practice of trainees before clinical exposure. However, current manikin-based training regimen is ineffective in providing satisfactory real-time procedural guidance for accurate assessment due to the lack of see-through visualization within the manikin. The training efficiency is further reduced by the limited availability of expert instructors, which inevitably results in a long learning curve for trainees. To this end, we propose an intelligent Augmented Reality (AR) training framework that provides trainees with a complete visualization of the ETI procedure for real-time guidance and assessment. Specifically, the proposed framework is capable of capturing the motions of the laryngoscope and the manikin and offer 3D see-through visualization rendered to the head-mounted display (HMD). Furthermore, an attention-based Convolutional Neural Network (CNN) model is developed to automatically assess the ETI performance from the captured motions as well as identify regions of motions that significantly contribute to the performance evaluation. Lastly, augmented user-friendly feedback is delivered with interpretable results with the ETI scoring rubric through the color-coded motion trajectory that classifies highlighted regions that need more practice. The classification accuracy of our machine learning model is 84.6%.
10 May 2018 in Computer Animation and Virtual Worlds
Purpose: The study aimed to develop a motion capture system that can track, visualize, and analyze the entire performance of self-injection with the auto-injector. Methods: Each of nine healthy subjects and 29 rheumatoid arthritic (RA) patients with different degrees of hand disability performed two simulated injections into an injection pad while six degrees of freedom (DOF) motions of the auto-injector and the injection pad were captured. We quantitatively measured the performance of the injection by calculating needle displacement from the motion trajectories. The max, mean, and SD of needle displacement were analyzed. Assessments of device acceptance and usability were evaluated by a survey questionnaire and independent observations of compliance with the device instruction for use (IFU). Results: A total of 80 simulated injections were performed. Our results showed a similar level of performance among all the subjects with slightly larger, but not statistically significant, needle displacement in the RA group. In particular, no significant effects regarding previous experience in self-injection, grip method, pain in hand, and Cochin score in the RA group were found to have an impact on the mean needle displacement. Moreover, the analysis of needle displacement for different durations of injections indicated that most of the subjects reached their personal maximum displacement in 15 seconds and remained steady or exhibited a small amount of increase from 15 to 60 seconds. Device acceptance was high for most of the questions (ie, >4; >80%) based on a 0–5-point scale or percentage of acceptance. The overall compliance with the device IFU was high for the first injection (96.05%) and reached 98.02% for the second injection. Conclusion: We demonstrated the feasibility of tracking the motions of injection to measure the performance of simulated self-injection. The comparisons of needle displacement showed that even RA patients with severe hand disability could properly perform self-injection with this auto-injector at a similar level with the healthy subjects. Finally, the observed high device acceptance and compliance with device IFU suggest that the system is convenient and easy to use.
The creation of personalized avatars that may be morphed to simulate realistic changes in body size is useful when studying self-perception of body size. One drawback is that these methods are resource intensive compared to rating scales that rely upon generalized drawings. Little is known about how body perception ratings compare across different methods, particularly across differing levels of personalized detail in visualizations. This knowledge is essential to inform future decisions about the appropriate tradeoff between personalized realism and resource availability. The current study aimed to determine the impact of varying degrees of personalized realism on self-perception of body size. We explored this topic in young adult women, using a generalized line drawing scale, as well as several types of personalized avatars, including 3D textured images presented in immersive virtual reality (VR). Body perception ratings using generalized line drawings were often higher than responses using individualized visualization methods. While the personalized details seemed to help with identification, there were few differences among the three conditions containing different amounts of individualized realism (e.g., photo-realistic texture). These results suggest that using scales based on personalized texture and limb dimensions are beneficial, although presentation in immersive VR may not be essential.
1 August 2021 in Journal of Biomedical Informatics
Neonatal endotracheal intubation (ETI) is an important, complex resuscitation skill, which requires a significant amount of practice to master. Current ETI practice is conducted on the physical manikin and relies on the expert instructors' assessment. Since the training opportunities are limited by the availability of expert instructors, an automatic assessment model is highly desirable. However, automating ETI assessment is challenging due to the complexity of identifying crucial features, providing accurate evaluations and offering valuable feedback to trainees. In this paper, we propose a dilated Convolutional Neural Network (CNN) based ETI assessment model, which can automatically provide an overall score and performance feedback to pediatric trainees. The proposed assessment model takes the captured kinematic multivariate time-series (MTS) data from the manikin-based augmented ETI system that we developed, automatically extracts the crucial features of captured data, and eventually provides an overall score as output. Furthermore, the visualization based on the class activation mapping (CAM) can automatically identify the motions that have significant impact on the overall score, thus providing useful feedback to trainees. Our model can achieve 92.2% average classification accuracy using the Leave-One-Out-Cross-Validation (LOOCV).