Multi-modal medical image registration takes an essential role in image-based clinical diagnosis and surgical planning. It is not trivial due to appearance variations across different modalities. Rigidly aligning two images is used to register rigid body structure, and it is also usually the first step for deformable registration with a large discrepancy. In the field of computer vision, one well-established method for image alignment is to find corresponding points from two images, and image alignment is based on identified corresponding points. Our method lies in this category. Feature representation is crucial in finding corresponding points. However, conventional feature representation like SIFT does not take multi-modal information into account, and thus, it fails. In this paper, we propose a Convolution Neural Network Feature-based Registration (CNNFR) method for aligning the multi-modal medical image. The important component in this method is learning keypoint descriptors using contrastive metric learning, which minimizes the difference between two feature representations from two corresponding points and maximizes difference of two feature representation from two distant points. Also, a transfer learning-based CNNRF (TrCNNRF) is proposed to improve the generalization learning performance when the training data are insufficient. Experimental results demonstrate that the proposed methods can achieve superior performance regarding both accuracy and robustness, which can be used to rigidly register multi-modal images and provide an initial estimation for non-rigid registration in clinical practices.

The rapid growth of social media has resulted in an explosion of online news content, leading to a significant increase in the spread of misleading or false information. While machine learning techniques have been widely applied to detect fake news, the scarcity of labeled datasets remains a critical challenge. Misinformation frequently appears as paired text and images, where a news article or headline is accompanied by a related visuals. In this paper, we introduce a self-learning multimodal model for fake news classification. The model leverages contrastive learning, a robust method for feature extraction that operates without requiring labeled data, and integrates the strengths of Large Language Models (LLMs) to jointly analyze both text and image features. LLMs are excel at this task due to their ability to process diverse linguistic data drawn from extensive training corpora. Our experimental results on a public dataset demonstrate that the proposed model outperforms several state-of-the-art classification approaches, achieving over 85% accuracy, precision, recall, and F1-score. These findings highlight the model's effectiveness in tackling the challenges of multimodal fake news detection.

Precipitation nowcasting plays an important role in mitigating the damage caused by severe weather. The objective of precipitation nowcasting is to forecast the weather conditions 0–2 h ahead. Traditional models based on numerical weather prediction and radar echo extrapolation obtain relatively better results. In recent years, models based on deep learning have also been applied to precipitation nowcasting and have shown improvement. However, the forecast accuracy is decreased with longer forecast times and higher intensities. To mitigate the shortcomings of existing models for precipitation nowcasting, we propose a novel model that fuses spatiotemporal features for precipitation nowcasting. The proposed model uses an encoder–forecaster framework that is similar to U-Net. First, in the encoder, we propose a spatial and temporal multi-head squared attention module based on MaxPool and AveragePool to capture every independent sequence feature, as well as a global spatial and temporal feedforward network, to learn the global and long-distance relationships between whole spatiotemporal sequences. Second, we propose a cross-feature fusion strategy to enhance the interactions between features. This strategy is applied to the components of the forecaster. Based on the cross-feature fusion strategy, we constructed a novel multi-head squared cross-feature fusion attention module and cross-feature fusion feedforward network in the forecaster. Comprehensive experimental results demonstrated that the proposed model more effectively forecasted high-intensity levels than other models. These results prove the effectiveness of the proposed model in terms of predicting convective weather. This indicates that our proposed model provides a feasible solution for precipitation nowcasting. Extensive experiments also proved the effectiveness of the components of the proposed model.