Surgical workflow and skill analysis are key technologies for the next generation of cognitive surgical assistance systems. These systems could increase the safety of the operation through context-sensitive warnings and semi-autonomous robotic assistance or improve training of surgeons via data-driven feedback. In surgical workflow analysis up to 91% average precision has been reported for phase recognition on an open data single-center video dataset. In this work we investigated the generalizability of phase recognition algorithms in a multicenter setting including more difficult recognition tasks such as surgical action and surgical skill. To achieve this goal, a dataset with 33 laparoscopic cholecystectomy videos from three surgical centers with a total operation time of 22 h was created. Labels included framewise annotation of seven surgical phases with 250 phase transitions, 5514 occurences of four surgical actions, 6980 occurences of 21 surgical instruments from seven instrument categories and 495 skill classifications in five skill dimensions. The dataset was used in the 2019 international Endoscopic Vision challenge, sub-challenge for surgical workflow and skill analysis. Here, 12 research teams trained and submitted their machine learning algorithms for recognition of phase, action, instrument and/or skill assessment. F1-scores were achieved for phase recognition between 23.9% and 67.7% (n = 9 teams), for instrument presence detection between 38.5% and 63.8% (n = 8 teams), but for action recognition only between 21.8% and 23.3% (n = 5 teams). The average absolute error for skill assessment was 0.78 (n = 1 team). Surgical workflow and skill analysis are promising technologies to support the surgical team, but there is still room for improvement, as shown by our comparison of machine learning algorithms. This novel HeiChole benchmark can be used for comparable evaluation and validation of future work. In future studies, it is of utmost importance to create more open, high-quality datasets in order to allow the development of artificial intelligence and cognitive robotics in surgery.

The early detection of glaucoma is essential in preventing visual impairment. Artificial intelligence (AI) can be used to analyze color fundus photographs (CFPs) in a cost-effective manner, making glaucoma screening more accessible. While AI models for glaucoma screening from CFPs have shown promising results in laboratory settings, their performance decreases significantly in real-world scenarios due to the presence of out-of-distribution and low-quality images. To address this issue, we propose the Artificial Intelligence for Robust Glaucoma Screening (AIROGS) challenge. This challenge includes a large dataset of around 113,000 images from about 60,000 patients and 500 different screening centers, and encourages the development of algorithms that are robust to ungradable and unexpected input data. We evaluated solutions from 14 teams in this paper and found that the best teams performed similarly to a set of 20 expert ophthalmologists and optometrists. The highest-scoring team achieved an area under the receiver operating characteristic curve of 0.99 (95% CI: 0.98-0.99) for detecting ungradable images on-the-fly. Additionally, many of the algorithms showed robust performance when tested on three other publicly available datasets. These results demonstrate the feasibility of robust AI-enabled glaucoma screening.

This paper proposes an automatic classification method based on machine learning in contrast-enhanced ultrasonography (CEUS) of focal liver lesions using the contrast agent Sonazoid. This method yields spatial and temporal features in the arterial phase, portal phase, and post-vascular phase, as well as max-hold images. The lesions are classified as benign or malignant and again as benign, hepatocellular carcinoma (HCC), or metastatic liver tumor using support vector machines (SVM) with a combination of selected optimal features. Experimental results using 98 subjects indicated that the benign and malignant classification has 94.0% sensitivity, 87.1% specificity, and 91.8% accuracy, and the accuracy of the benign, HCC, and metastatic liver tumor classifications are 84.4%, 87.7%, and 85.7%, respectively. The selected features in the SVM indicate that combining features from the three phases are important for classifying FLLs, especially, for the benign and malignant classifications. The experimental results are consistent with CEUS guidelines for diagnosing FLLs. This research can be considered to be a validation study, that confirms the importance of using features from these phases of the examination in a quantitative manner. In addition, the experimental results indicate that for the benign and malignant classifications, the specificity without the post-vascular phase features is significantly lower than the specificity with the post-vascular phase features. We also conducted an experiment on the operator dependency of setting regions of interest and observed that the intra-operator and inter-operator kappa coefficients were 0.45 and 0.77, respectively.

Video anomaly detection (VAD) is crucial for intelligent video surveillance systems. Many VAD methods for crowds in public spaces have been evaluated using videos captured by fixed cameras. Recently, drones have also been used for video surveillance, but the datasets of human activity captured by drones are relatively small or include actions by only a small number of people. Therefore, we created a large-scale dataset called Crowd Anomaly detection from Drone and Ground (CADG). A moving drone and two fixed ground cameras captured 448×3 clips, comprising 236K×3 frames, of five types of crowd activities crucial to video surveillance. We also propose a unified learning framework for simultaneous anomaly detection and classification using a simple yet strong and computationally inexpensive temporal encoder. Experiments on the dataset show that the method detects crowd anomalies more accurately than conventional methods, but still highlight the difficulties of stable VAD from a moving drone and bridging the domain gap between views. The dataset is available on the project page at https://cadg24.github.io/home.