Pressure injury management requires coordinated decisions on wound staging, debridement, and dressing selection, yet most computational studies focus on isolated tasks using a single data modality. This study developed a multimodal, multi-task framework integrating wound photographs and clinical text to support pressure injury stage classification, debridement necessity prediction, primary dressing recommendation, and secondary dressing recommendation. Models were trained on an expert-annotated public dataset and externally validated using an independent private cohort. We systematically compared image-only models, text-only models, large language model-based approaches, conventional image–text fusion strategies, and image–language model combinations. Among the evaluated models, ResNet-50 combined with DeepSeek-R1-Distill-Qwen-1.5B showed the best overall balance across classification performance, text-generation quality, and validation robustness. On the validation set, the final model achieved 82.92% accuracy, 81.26% macro-F1, and 99.01% area under the curve for stage classification, and 89.91% accuracy, 89.83% macro-F1, and 96.24% area under the curve for debridement prediction. For dressing recommendation, the model achieved ROUGE-1 scores of 58.30% and 61.06% for primary and secondary dressings, respectively. Interpretability analyses indicated clinically relevant image attention and text attribution patterns. These findings suggest that multimodal learning may provide a clinically aligned decision-support approach for pressure injury assessment and treatment recommendations.
Functional brain network has been widely studied in many previous work for brain disorder diagnosis and brain network analysis. However, most previous work focus on static dynamic brain network research. Lots of recent work reveals that the brain shows dynamic activity even in resting state. Such dynamic brain functional connectivity reveals discriminative patterns for identifying many brain disorders. Current sliding window based dynamic brain connectivity framework are not easy to be applied to real clinical applications due to many issues: First, how to set up the optimal sliding window size and how to determine the threshold for the brain connectivity patterns. Secondly, how to represent the high dimensional dynamic brain connectivity pattern in a low dimensional representations for diagnosis purpose. Last, how to deal with the different length dynamic brain network patterns especially when the raw data are of different length. In order to address all those above issues, we proposed a new framework, which employs multiple scale sliding windows and automatically learns a sparse and low ran dynamic brain functional connectivity patterns from raw fMRI data. Furthermore, we are able to measure different length dynamic brain functional connectivity patterns in an equal space by learning a sparse coded convolutional filters. We have evaluated our method with state of the art dynamic brain network methods and the results demonstrated the strong potential of our methods for brain disorder diagnosis in real clinical applications.
We present a new task of query auto-completion for estimating instance probabilities. We complete a user query prefix conditioned upon an image. Given the complete query, we fine tune a BERT embedding for estimating probabilities of a broad set of instances. The resulting instance probabilities are used for selection while being agnostic to the segmentation or attention mechanism. Our results demonstrate that auto-completion using both language and vision performs better than using only language, and that fine tuning a BERT embedding allows to efficiently rank instances in the image. In the spirit of reproducible research we make our data, models, and code available.