Multimodal medical question answering (MMQA) is a vital area bridging healthcare and Artificial Intelligence (AI). This survey methodically examines the MMQA research published in recent years. We collect academic literature through Google Scholar, applying bibliometric analysis to the publications and datasets used in these studies. Our analysis uncovers the increasing interest in MMQA over time, with diverse domains such as natural language processing, computer vision, and large language models contributing to the research. The AI methods used in multimodal question answering in the medical domain are a prominent focus, accompanied by applicability of MMQA to the medical field. MMQA in the medical field has its unique challenges due to the sensitive nature of medicine as a science dealing with human health. The survey reveals MMQA research to be in an exploratory stage, discussing different methods, datasets, and potential business models. Future research is expected to focus on application development by big tech companies, such as MedPalm. The survey aims to provide insights into the current state of multimodal medical question answering, highlighting the growing interest from academia and industry. The identified research gaps and trends will guide future investigations and encourage collaborative efforts to advance this transformative field.

This study explores the potential of natural language models, including large language models, to extract causal relations from medical texts, specifically from clinical practice guidelines (CPGs). The outcomes of causality extraction from clinical practice guidelines for gestational diabetes are presented, marking a first in the field. The results are reported on a set of experiments using variants of BERT (BioBERT, DistilBERT, and BERT) and using newer large language models (LLMs), namely, GPT-4 and LLAMA2. Our experiments show that BioBERT performed better than other models, including the large language models, with an average F1-score of 0.72. The GPT-4 and LLAMA2 results show similar performance but less consistency. The code and an annotated corpus of causal statements within the clinical practice guidelines for gestational diabetes are released. Extracting causal structures might help identify LLMs’ hallucinations and possibly prevent some medical errors if LLMs are used in patient settings. Some practical extensions of extracting causal statements from medical text would include providing additional diagnostic support based on less frequent cause–effect relationships, identifying possible inconsistencies in medical guidelines, and evaluating the evidence for recommendations.

Topological Data Analysis (TDA) refers to a collection of methods that find the structure of shapes in data. Although recently, TDA methods have been used in many areas of data mining, it has not been widely applied to text mining tasks. In most text processing algorithms, the order in which different entities appear or co-appear is being lost. Assuming these lost orders are informative features of the data, TDA may play a significant role in the resulted gap on text processing state of the art. Once provided, the topology of different entities through a textual document may reveal some additive information regarding the document that is not reflected in any other features from conventional text processing methods. In this paper, we introduce a novel approach that hires TDA in text processing in order to capture and use the topology of different same-type entities in textual documents. First, we will show how to extract some topological signatures in the text using persistent homology-i.e., a TDA tool that captures topological signature of data cloud. Then we will show how to utilize these signatures for text classification.

This article presents a state-of-the-art system to extract and synthesize causal statements from company reports into a directed causal graph. The extracted information is organized by its relevance to different stakeholder group benefits (customers, employees, investors, and the community/environment). The presented method of synthesizing extracted data into a knowledge graph comprises a framework that can be used for similar tasks in other domains, e.g., medical information. The current work addresses the problem of finding, organizing, and synthesizing a view of the cause-and-effect relationships based on textual data in order to inform and even prescribe the best actions that may affect target business outcomes related to the benefits for different stakeholders (customers, employees, investors, and the community/environment).

Universities typically offer residential students a variety of fast-food dining options as part of the student meal plan. When residential students make fast-food purchases on campus there is a digital record of the transaction which can be used to study food purchasing behavior. This study examines the association between student demographic, economic, and behavioral factors and the healthfulness of student fast-food purchases. The 3781 fast-food items sold at the University of North Carolina at Charlotte from fall 2016 to spring 2019 were given a Fast-Food Health Score. Each student participating in the university meal plan was given a Student Average Fast-Food Health Score; calculated by averaging the Fast-Food Health Scores associated with each food and beverage item the student purchased at a fast-food vendor, concession stand, or convenience store over a semester. This analysis included 14,367 students who generated 1,593,235 transactions valued at $10,757,110. Multivariate analyses were used to examine demographic, economic, and behavioral factors associated with Student Average Fast-Food Health Scores. Being of a low income, spending more money on fast-food items, and having a lower GPA were associated with lower Student Average Fast-Food Health Scores. Future research utilizing institutional food transaction data to study healthy food choices is warranted.