The disease and pest recognition algorithms based on computer vision can automatically process and analyze a large amount of disease and pest images, thereby achieving rapid and accurate identification of disease and pest categories on crop leaves. Currently, most studies use deep learning models for feature extraction and identification of crop leaf disease and pest images. However, these methods are often seen as “black box” model, making it difficult to interpret the basis for their specific decisions. To address this issue, we propose an intrinsically interpretable crop leaf disease and pest identification model named C ontrastive P rototypical Part Net work (CPNet). The idea of CPNet is to find the key regions that influence the model’s decision by calculating the similarity values between the convolutional feature maps and the learnable latent prototype feature representations. Moreover, because the limited availability of data resources for crop leaf disease and pest images, we employ a supervised contrastive learning strategy to capture the similar information between examples in one class and contrast them with examples in other classes. Finally, we evaluate our approach on four publicly available datasets, and the experimental results demonstrate that our proposed CPNet not only achieves improvements in performance over baseline methods across multiple datasets, but also provides interpretable evidence for crop leaf disease and pest identification.

Recent advances in sentiment analysis have primarily focused on fusing multimodal information from video data, including visual, acoustic, and textual features, across temporal sequences. While great effort has been made to integrate or fuse information across modalities, less is known about the extent to which temporal segments contribute to model decisions. In addition, current interpretable methods, such as prototype networks, are primarily designed for uni-modal analysis and fail to handle the complex interactions between multiple modalities and temporal dependencies inherent in video data. To address the challenges, we propose M ulti M odal P rototypical Net works (MMPNet), which extends prototype-based interpretability to multimodal sentiment classification. Specifically, MMPNet can identify contributions of time-level features and leverage them to explain why a particular prediction was made, while also helping to find the relative importance of modality-level features. Experimental results show that MMPNet outperforms existing methods by 2.9% and 1.6% in accuracy on CMU-MOSI and CMU-MOSEI respectively, and achieves better interpretability.

As online video news gains more popularity for information acquisition, the presence of fake news video poses a new threat to users seeking the real news. Though previous studies have made much progress in detection fake news in text and image formats, video-formed fake news brings new and unique challenges: 1) the modality heterogeneity of fake news video, 2) the inherent data non-alignment of different modalities. However, existing methods simply fuse the multimodal feature at the decision layer or feature layer, and fail to explore the intrinsic complex correlations between different modalities. In this paper, we propose a multimodal graph contrastive learning framework which learning complex relations between different modalities for detecting fake news video. Specifically, to solve the issue of modality heterogeneity, we construct unimodal homogeneous graphs, cross-modal heterogeneous graphs which model the hidden relations within and across modalities, respectively. We design graph contrastive learning module to obtain graph representation without explicitly aligning the data which captures the underlying correlation across modalities. We conduct experiments on the public dataset and the results show that our proposed model outperforms existing methods on fake news video detection.

Deep learning provides new methods for crop pest and disease identification and control, offering unique advantages in terms of recognition accuracy and efficiency. However, deep learning models generally lack interpretability, and their internal decision-making processes are difficult to understand. This, to some extent, undermines users’ trust in the model’s predictions and hinders its large-scale application in agricultural production. Therefore, improving model transparency and interpretability has become an important research direction. To address this issue, this study proposes a novel interpretable crop pest and disease identification model, the Contrastive Prototype Tree (CPTR). The model is designed around the core structure of “concept prototypes and decision tree,” which builds clear prototype matching paths for each recognition result. This enables the model to not only have strong classification capability but also provide intuitive explanations. Additionally, the study introduces the SimCLR contrastive learning framework to enhance the model’s ability to express deep image features. SimCLR guides the model to learn more discriminative visual features by maximizing the similarity between positive sample pairs and minimizing the similarity between negative sample pairs, thereby improving overall recognition performance. This study evaluated the model on three datasets: AppleLeaf9, Cassava, and Cashew. The experimental results show that CPTR achieves accuracies of 83.74%, 94.80%, and 96.01% on the three datasets, representing improvements of 4.12%, 0.34%, and 0.51% compared to Prototype Tree, respectively. These results indicate that the proposed model achieves the highest accuracy across different datasets, demonstrating its effectiveness.