Accurate identification of tea diseases and pests is a key challenge in smart agriculture. Current approaches to tea disease and pest identification suffer from a scarcity of high-quality annotated image data and poor generalization of existing models in real-world field environments. To address these issues, this paper first constructs and releases a dataset of images of tea diseases and pests captured in real-world field scenarios. The dataset uses leaf-level annotations and covers six common tea disease and pest categories in Guizhou Province, China. It contains 549 high-resolution images covering varying lighting conditions, backgrounds, and disease severity levels. Based on this dataset, we propose a convolutional neural network model named GatedFCA-YOLO, which integrates a small-object detection layer with an adaptive attention mechanism. Specifically, the small-object detection layer preserves high-resolution details, effectively improving recall of minute lesions. Meanwhile, the GatedFCA module is designed to fuse a spatial gating mechanism with FCAttention. It enables adaptive feature enhancement and significantly boosts the model’s recognition robustness under complex backgrounds. Experimental results on our dataset show that GatedFCA-YOLO achieves 78.9% mAP@0.5, which is 3% increased compared to the baseline model YOLO11n, thereby verifying the effectiveness of the proposed method.