To address the challenges associated with kiwi fruit detection methods, such as low average accuracy, inaccurate recognition of fruits, and long recognition time, this study proposes a novel kiwi fruit recognition method based on an improved lightweight network S-YOLOv4-tiny detection algorithm. Firstly, the YOLOv4-tiny algorithm utilizes the CSPdarknet53-tiny network as a backbone feature extraction network, replacing the CSPdarknet53 network in the YOLOv4 algorithm to enhance the speed of kiwi fruit recognition. Additionally, a squeeze-and-excitation network has been incorporated into the S-YOLOv4-tiny detection algorithm to improve accurate image extraction of kiwi fruit characteristics. Finally, enhancing dataset pictures using mosaic methods has improved precision in the characteristic recognition of kiwi fruits. The experimental results demonstrate that the recognition and positioning of kiwi fruits have yielded improved outcomes. The mean average precision (mAP) stands at 89.75%, with a detection precision of 93.96% and a single-picture detection time of 8.50 ms. Compared to the YOLOv4-tiny detection algorithm network, the network in this study exhibits a 7.07% increase in mean average precision and a 1.16% acceleration in detection time. Furthermore, an enhancement method based on the Squeeze-and-Excitation Network (SENet) is proposed, as opposed to the convolutional block attention module (CBAM) and efficient channel attention (ECA). This approach effectively addresses issues related to slow training speed and low recognition accuracy of kiwi fruit, offering valuable technical insights for efficient mechanical picking methods.