Smart agriculture has become an inevitable trend in the development of modern agriculture, especially promoted by the continuous progress of large language models like chat generative pre-trained transformer (ChatGPT) and general language model (ChatGLM). Although these large models perform well in general knowledge learning, they still have certain limitations and errors when facing agricultural professional knowledge about crop disease identification, growth stage judgment, and so on. Agricultural data involves images and texts and other modalities, which play an important role in agricultural production and management. In order to better learn the characteristics of different modal data in agriculture, realize cross-modal data fusion, and thus understand complex application scenarios, we propose a framework AgriVLM that uses a large amount of agricultural data to fine-tune the visual language model to analyze agricultural data. It can fuse multimodal data and provide more comprehensive agricultural decision support. Specifically, it utilizes Q-former as a bridge between an image encoder and a language model to achieve a cross-modal fusion of agricultural images and text data. Then, we apply a Low-Rank adaptive to fine-tune the language model to achieve an alignment between agricultural image features and a pre-trained language model. The experimental results prove that AgriVLM demonstrates great performance in crop disease recognition and growth stage recognition, with recognition accuracy exceeding 90%, demonstrating its capability to analyze different modalities of agricultural data.
The automated detection of personal protective equipment (PPE) is crucial for ensuring operational safety in hazardous environments. However, background interference and target semantic ambiguity caused by dramatic illumination changes and occlusions pose severe challenges to traditional deep learning models. To address these challenges, we present TGF-ChD, a detection model based on the text-guided fusion (TGF) module, which enhances feature representation by introducing category semantic priors. TGF-ChD first employs text-guided attention to spatially capture significant regions based on semantic constraints; subsequently, through semantic feature modulation, which calibrates channel gains and shifts according to semantic relevance, high-level semantic information is deeply embedded into multi-scale visual features. On the Chemical Industry PPE dataset, the model achieved an mAP of 54.3% and an mAP50 of 84.9%, exceeding TOOD by 2.0 and 6.0 percentage points, respectively. Furthermore, the model achieves improved performance on challenging categories, such as ‘blurred clothing’ and ‘blurred heads’ caused by severe reflections. These results indicate that jointly applying spatial and channel-level semantic conditioning can improve PPE detection under the evaluated complex industrial conditions.