In view of the continuous increase in labor costs for complex picking tasks, there is an urgent demand for intelligent harvesting robots in the global fresh fruit cultivation industry. Fruit visual information is essential to guide robotic harvesting. However, obtaining accurate visual information about the target is critical in complex agricultural environments. The main challenges include the image color distortion under changeable natural light, occlusions from the interlaced plant organs (stems, leaves, and fruits), and the picking point location on fruits with variable shapes and poses. On top of summarizing the current status of typical fresh fruit harvesting robots, this paper outlined the state-of-the-art advance of visual information acquisition technology, including image acquisition in the natural environment, fruit recognition from the complex backgrounds, target stereo locating and measurement, and fruit search among the plants. It then analyzed existing problems and raised future potential research trends from two aspects, multiple images fusion and self-improving algorithm model.

Conventional mobile robots employ LIDAR for indoor global positioning and navigation, thus having strict requirements for the ground environment. Under the complicated ground conditions in the greenhouse, the accumulative error of odometer (ODOM) that arises from wheel slip is easy to occur during the long-time operation of the robot, which decreases the accuracy of robot positioning and mapping. To solve the above problem, an integrated positioning system based on UWB (ultra-wideband)/IMU (inertial measurement unit)/ODOM/LIDAR is proposed. First, UWB/IMU/ODOM is integrated by the Extended Kalman Filter (EKF) algorithm to obtain the estimated positioning information. Second, LIDAR is integrated with the established two-dimensional (2D) map by the Adaptive Monte Carlo Localization (AMCL) algorithm to achieve the global positioning of the robot. As indicated by the experiments, the integrated positioning system based on UWB/IMU/ODOM/LIDAR effectively reduced the positioning accumulative error of the robot in the greenhouse environment. At the three moving speeds, including 0.3 m/s, 0.5 m/s, and 0.7 m/s, the maximum lateral error is lower than 0.1 m, and the maximum lateral root mean square error (RMSE) reaches 0.04 m. For global positioning, the RMSEs of the x-axis direction, the y-axis direction, and the overall positioning are estimated as 0.092, 0.069, and 0.079 m, respectively, and the average positioning time of the system is obtained as 72.1 ms. This was sufficient for robot operation in greenhouse situations that need precise positioning and navigation.

Intermittent stop–move motion planning is essential for optimizing the efficiency of harvesting robots in greenhouse settings. Addressing issues like frequent stops, missed targets, and uneven task allocation, this study introduced a novel intermittent motion planning model using deep reinforcement learning for a dual-arm harvesting robot vehicle. Initially, the model gathered real-time coordinate data of target fruits on both sides of the robot, and projected these coordinates onto a two-dimensional map. Subsequently, the DDPG (Deep Deterministic Policy Gradient) algorithm was employed to generate parking node sequences for the robotic vehicle. A dynamic simulation environment, designed to mimic industrial greenhouse conditions, was developed to enhance the DDPG to generalize to real-world scenarios. Simulation results have indicated that the convergence performance of the DDPG model was improved by 19.82% and 33.66% compared to the SAC and TD3 models, respectively. In tomato greenhouse experiments, the model reduced vehicle parking frequency by 46.5% and 36.1% and decreased arm idleness by 42.9% and 33.9%, compared to grid-based and area division algorithms, without missing any targets. The average time required to generate planned paths was 6.9 ms. These findings demonstrate that the parking planning method proposed in this paper can effectively improve the overall harvesting efficiency and allocate tasks for a dual-arm harvesting robot in a more rational manner.

Accurate identification of field pests has crucial decision-making significance for integrated pest control. Most current research focuses on the identification of pests on the sticky card or the case of great differences between the target and the background. There is little research on field pest identification with protective color characteristics. Aiming at the problem that it is difficult to identify pests with protective color characteristics in the complex field environment, a field pest identification method based on near-infrared imaging technology and YOLOv5 is proposed in this paper. Firstly, an appropriate infrared filter and ring light source have been selected to build an image acquisition system according to the wavelength with the largest spectral reflectance difference between the spectral curves of the pest (Pieris rapae) and its host plants (cabbage), which are formed by specific spectral characteristics. Then, field pest images have been collected to construct a data set, which has been trained and tested through YOLOv5. Experimental results demonstrate that the average time required to detect one pest image is 0.56 s, and the mAP reaches 99.7%.

Dexterous manipulation and gradual placement are crucial for preserving fruit integrity during harvesting. Addressing the limitations of conventional path planning methods in learning manual compliant skills, we propose a novel method for tomato bunch collection that integrates human-robot skill transfer with Deep Deterministic Policy Gradient (DDPG). In our method, a demonstrator manually guided the robotic arm using an existing tomato collection mechanism, with spatial trajectories recorded as demonstration paths. We then developed an enhanced DDPG-Z model that incorporates human skill replay for pre-training, expert reward regression loss to stabilize pre-training, and dynamic step-length returns to balance short- and long-term rewards. Subsequently, the agent was trained to minimize the deviations of key points between the demonstration paths and actual paths, effectively approximating human operations. In a highly realistic simulation environment, our method achieved a 25% improvement in convergence speed, a 10.3% increase in post-convergence reward, and a 51.3% boost in destination accuracy compared to the case without the demonstrations, whereas classical models such as DDPG, SAC (Soft Actor-Critic), and TD3 (Twin Delayed Deep Deterministic Policy Gradient) failed to converge within the prescribed episodes. This work provides valuable insights for enhancing the compliant operational performance of agricultural robots.