14 February 2024 in Artificial Intelligence in Medicine
Machine Learning models need large amounts of annotated data for training. In the field of medical imaging, labeled data is especially difficult to obtain because the annotations have to be performed by qualified physicians. Natural Language Processing (NLP) tools can be applied to radiology reports to extract labels for medical images automatically. Compared to manual labeling, this approach requires smaller annotation efforts and can therefore facilitate the creation of labeled medical image data sets. In this article, we summarize the literature on this topic spanning from 2013 to 2023, starting with a meta-analysis of the included articles, followed by a qualitative and quantitative systematization of the results. Overall, we found four types of studies on the extraction of labels from radiology reports: those describing systems based on symbolic NLP, statistical NLP, neural NLP, and those describing systems combining or comparing two or more of the latter. Despite the large variety of existing approaches, there is still room for further improvement. This work can contribute to the development of new techniques or the improvement of existing ones.
22 March 2013 in Journal of the American Society for Information Science and Technology
5 June 2017 in Journal of the Association for Information Science and Technology
29 November 2011 in Online Information Review
Purpose: The intent of this work is to evaluate several generalist and health‐specific search engines for retrieval of health information by consumers: to compare the retrieval effectiveness of these engines for different types of clinical queries, medical specialties and condition severity; and to compare the use of evaluation metrics for binary relevance scales and for graded ones.Design/methodology/approach: The authors conducted a study in which users evaluated the relevance of documents retrieved by four search engines for two different health information needs. Users could choose between generalist (Bing, Google, Sapo and Yahoo!) and health‐specific (MedlinePlus, SapoSaúde and WebMD) search engines. The authors then analysed the differences between search engines and groups of information needs with six different measures: graded average precision (gap), average precision (ap), gap@5, gap@10, ap@5 and ap@10.Findings: The results show that generalist web search engines surpass the precision of health‐specific engines. Google has the best performance, mainly in the top ten results. It was found that information needs associated with severe conditions are associated with higher precision, as are overview and psychiatry questions.Originality/value: The study is one of the first to use a recently proposed measure to evaluate the effectiveness of retrieval systems with graded relevance scales. It includes tasks from several medical specialties, types of clinical questions and different levels of severity which, to the best of the authors' knowledge, has not been done before. Moreover, users have considerable involvement in the experiment. The results help in understanding how search engines differ in their responses to health information needs, what types of online health information are more common on the web and how to improve this type of search.