Camellia oleifera is a crop of high economic value, yet it is particularly susceptible to various diseases and pests that significantly reduce its yield and quality. Consequently, the precise segmentation and classification of diseased Camellia leaves are vital for managing pests and diseases effectively. Deep learning exhibits significant advantages in the segmentation of plant diseases and pests, particularly in complex image processing and automated feature extraction. However, when employing single-modal models to segment Camellia oleifera diseases, three critical challenges arise: (A) lesions may closely resemble the colors of the complex background; (B) small sections of diseased leaves overlap; (C) the presence of multiple diseases on a single leaf. These factors considerably hinder segmentation accuracy. A novel multimodal model, CNN–Transformer Dual U-shaped Network (CTDUNet), based on a CNN–Transformer architecture, has been proposed to integrate image and text information. This model first utilizes text data to address the shortcomings of single-modal image features, enhancing its ability to distinguish lesions from environmental characteristics, even under conditions where they closely resemble one another. Additionally, we introduce Coordinate Space Attention (CSA), which focuses on the positional relationships between targets, thereby improving the segmentation of overlapping leaf edges. Furthermore, cross-attention (CA) is employed to align image and text features effectively, preserving local information and enhancing the perception and differentiation of various diseases. The CTDUNet model was evaluated on a self-made multimodal dataset compared against several models, including DeeplabV3+, UNet, PSPNet, Segformer, HrNet, and Language meets Vision Transformer (LViT). The experimental results demonstrate that CTDUNet achieved an mean Intersection over Union (mIoU) of 86.14%, surpassing both multimodal models and the best single-modal model by 3.91% and 5.84%, respectively. Additionally, CTDUNet exhibits high balance in the multi-class segmentation of Camellia oleifera diseases and pests. These results indicate the successful application of fused image and text multimodal information in the segmentation of Camellia disease, achieving outstanding performance.

Accurate pear sorting plays a crucial role in ensuring the quality of pears and increasing the sales of them. In the domain of intelligent pear sorting, precise target detection of pears is imperative. However, practical implementation faces challenges in achieving adequate accuracy in pear target detection due to the limitations of computational resources in embedded devices and the occurrence of occlusion among pears. To solve this problem, we built an image acquisition system based on pear sorting equipment and created a pear dataset containing 34,598 pear images under laboratory conditions. The dataset was meticulously annotated using the LabelImg software, resulting in a total of 154,688 precise annotations for pears, pear stems, pear calyxes, and pear defects. Furthermore, we propose an Extremely Compressed Lightweight Model for Pear Object Detection (ECLPOD) based on YOLOv7’s pipeline to assist in the pear sorting task. Firstly, the Hierarchical Interactive Shrinking Network (HISNet) was proposed, which contributed to efficient feature extraction with a limited amount of computation and parameters. The Bulk Feature Pyramid (BFP) module was then proposed to enhance pear contour information extraction during feature fusion. Finally, the Accuracy Compensation Strategy (ACS) was proposed to improve the detection capability of the model, especially for identification of the calyces and stalks of pears. The experimental results indicate that the ECLPOD achieves 90.1% precision (P) and 85.52% mAP⁵⁰ with only 0.58 million parameters and 1.3 GFLOPs of computation in the homemade pear dataset in this paper. Compared with YOLOv7, the number of parameters and the amount of computation for the ECLPOD are compressed to 1.5% and 1.3%, respectively. Compared with other mainstream methods, the ECLPOD achieves an optimal trade-off between accuracy and complexity. This suggests that the ECLPOD is superior to these existing approaches in the field of object detection for assisting pear sorting tasks with good potential for embedded device deployment.