Real-time global event detection particularly catastrophic events has benefited significantly due to the ubiquitous adoption of social media platforms and advancements in image classification and natural language processing. Social media is a rich repository of multimedia content during disasters, encompassing reports on casualties, infrastructure damage, and information about missing individuals. While previous research has predominantly concentrated on textual or image analysis, the proposed study presents a multimodal middle fusion paradigm that includes Cross-modal attention and Self-attention to improve learning from both image and text modalities. Through rigorous experimentation, we validate the effectiveness of our proposed middle fusion paradigm in leveraging complementary information from both textual and visual sources.The proposed intermediate design outperforms current late and early fusion structures, achieving an accuracy of 91.53% and 91.07% in the informativeness and disaster type recognition categories, respectively. This study is among the few that examine all three tasks in the CrisisMMD dataset by combining textual and image analysis, demonstrating an approximate improvement of about 2% in prediction accuracy compared to similar studies on the same dataset.Additionally, ablation studies indicate that it outperforms the best-selected unimodal classifiers, with a 3-5% increase in prediction accuracies across various tasks. Thus, the method aims to bolster emergency response capabilities by offering more precise insights into evolving events.

Background/Introduction: Skin lesion classification poses a critical diagnostic challenge in dermatology, where early and accurate identification has a direct impact on patient outcomes. While deep learning approaches have shown promise using dermatoscopic images alone, the integration of clinical metadata remains underexplored despite its potential to enhance diagnostic accuracy.Methods: We developed a novel multimodal data fusion framework that systematically integrates dermatoscopic images with clinical metadata for the classification of skin lesions. Using the HAM10000 dataset, we evaluated multiple fusion strategies, including simple concatenation, weighted concatenation, self-attention mechanisms, and cross-attention fusion. Clinical features were processed through a customized Multi-Layer Perceptron (MLP), while images were analyzed using a modified Residual Networks (ResNet) architecture. Model interpretability was enhanced using Gradient-weighted Class Activation Mapping (Grad-CAM) visualization to identify the contribution of clinical attributes to classification decisions.Results: Cross-attention fusion achieved the highest classification accuracy, demonstrating superior performance compared to unimodal approaches and simpler fusion techniques. The multimodal framework significantly outperformed image-only baselines, with cross-attention effectively capturing inter-modal dependencies and contextual relationships between visual and clinical data modalities.Discussion/Conclusions: Our findings demonstrate that integrating clinical metadata with dermatoscopic images substantially improves the accuracy of skin lesion classification. However, challenges, including class imbalance and the computational complexity of advanced fusion methods, require further investigation.

This study aimed to develop an advanced ensemble approach for automated classification of mental health disorders in social media posts. The research question was: can an ensemble of fine‐tuned transformer models (XLNet, RoBERTa, and ELECTRA) with Bayesian hyperparameter optimization improve the accuracy of mental health disorder classification in social media text. Three transformer models (XLNet, RoBERTa, and ELECTRA) were fine‐tuned on a dataset of social media posts labelled with 15 distinct mental health disorders. Bayesian optimization was employed for hyperparameter tuning, optimizing learning rate, number of epochs, gradient accumulation steps, and weight decay. A voting ensemble approach was then implemented to combine the predictions of the individual models. The proposed voting ensemble achieved the highest accuracy of 0.780, outperforming the individual models: XLNet (0.767), RoBERTa (0.775), and ELECTRA (0.755). The proposed ensemble approach, integrating XLNet, RoBERTa, and ELECTRA with Bayesian hyperparameter optimization, demonstrated improved accuracy in classifying mental health disorders from social media posts. This method shows promise for enhancing digital mental health research and potentially aiding in early detection and intervention strategies. Future work should focus on expanding the dataset, exploring additional ensemble techniques, and investigating the model's performance across different social media platforms and languages.