BACKGROUND: The markup of medical image datasets is based on the subjective interpretation of the observed entities by radiologists. There is currently no widely accepted protocol for determining ground truth based on radiologists reports. AIM: To assess the accuracy of radiologist interpretations and their agreement for the publicly available dataset CTLungCa-500, as well as the relationship between these parameters and the number of independent readers of CT scans. MATERIALS AND METHODS: Thirty-four radiologists took part in the dataset markup. The dataset included 536 patients who were at high risk of developing lung cancer. For each scan, six radiologists worked independently to create a report. After that, an arbitrator reviewed the lesions discovered by them. The number of true-positive, false-positive, true-negative, and false-negative findings was calculated for each reader to assess diagnostic accuracy. Further, the inter-observer variability was analyzed using the percentage agreement metric. RESULTS: An increase in the number of independent readers providing CT scan interpretations leads to accuracy increase associated with a decrease in agreement. The majority of disagreements were associated with the presence of a lung nodule in a specific site of the CT scan. CONCLUSION: If arbitration is provided, an increase in the number of independent initial readers can improve their combined accuracy. The experience and diagnostic accuracy of individual readers have no bearing on the quality of a crowd-tagging annotation. At four independent readings per CT scan, the optimal balance of markup accuracy and cost was achieved.

Introduction. Medical imaging techniques can diagnose many diseases at the early stages of their development, improving the patient survival. Artificial intelligence (AI) systems, requiring the high-quality annotated and marked-up sets of medical images, are a suitable and promising means of improving the diagnostics’ quality. The purpose of the study was to develop a methodology and software for creating AIS training sets. Material and methods. We compared the main annotation methods’ performance and accuracy and based the information system on the most efficient method in both domains to develop an optimal approach. To markup objects of interest, we used the cluster model of lesions localization previously developed by the authors. We used C++ and Kotlin programming languages for software development. Results. A structured annotation template with delivered a glossary of terms became the basis of the information system. The latter consists of three interacting modules, two of which are executed on a remote server’s capacities and one on a personal computer or mobile device of the end-user. The first module is a web service responsible for the workflow logic. The second module, a web server, is responsible for interacting with client applications. Its role is to identify users and manage the database and Picture Archiving and Communication System (PACS) connections. The front-end module is a web application with a graphical interface that assists the end-user in images’ markup and annotation. Conclusions. An algorithmic basis and a software package have been created for annotation and markup of CT images. The resulting information system was used in a large-scale lung cancer screening project for the creation of medical imaging datasets.

Computed tomography (CT) has been an essential diagnostic tool during the COVID-19 pandemic. The study aimed to develop an optimal CT protocol in terms of safety and reliability. For this, we assessed the inter-observer agreement between CT and low-dose CT (LDCT) with soft and sharp kernels using a semi-quantitative severity scale in a prospective study (Moscow, Russia). Two consecutive scans with CT and LDCT were performed in a single visit. Reading was performed by ten radiologists with 3–25 years’ experience. The study included 230 patients, and statistical analysis showed LDCT with a sharp kernel as the most reliable protocol (percentage agreement 74.35 ± 43.77%), but its advantage was marginal. There was no significant correlation between radiologists’ experience and average percentage agreement for all four evaluated protocols. Regarding the radiation exposure, CTDI_vol was 3.6 ± 0.64 times lower for LDCT. In conclusion, CT and LDCT with soft and sharp reconstructions are equally reliable for COVID-19 reporting using the “CT 0-4” scale. The LDCT protocol allows for a significant decrease in radiation exposure but may be restricted by body mass index.

Information in industry, research, and the public sector is widely stored as rendered documents (e.g., PDF files, scans). Hence, to enable downstream tasks, systems are needed that map rendered documents onto a structured hierarchical format. However, existing systems for this task are limited by heuristics and are not end-to-end trainable. In this work, we introduce the Document Structure Generator (DSG), a novel system for document parsing that is fully end-to-end trainable. DSG combines a deep neural network for parsing (i) entities in documents (e.g., figures, text blocks, headers, etc.) and (ii) relations that capture the sequence and nested structure between entities. Unlike existing systems that rely on heuristics, our DSG is trained end-to-end, making it effective and flexible for real-world applications. We further contribute a new, large-scale dataset called E-Periodica comprising real-world magazines with complex document structures for evaluation. Our results demonstrate that our DSG outperforms commercial OCR tools and, on top of that, achieves state-of-the-art performance. To the best of our knowledge, our DSG system is the first end-to-end trainable system for hierarchical document parsing.