With advances in precision agriculture, autonomous agricultural machines can reduce human labor, optimize workflow, and increase productivity. Accurate and reliable obstacle-detection and avoidance systems are essential for ensuring the safety of automated agricultural machines. Existing LiDAR-based obstacle detection methods for the farmland environment process the point clouds via manually designed features, which is time-consuming, labor-intensive, and weak in terms of generalization. In contrast, deep learning has a powerful ability to learn features autonomously. In this study, we attempted to apply deep learning in LiDAR-based 3D obstacle detection for the farmland environment. In terms of perception hardware, we established a data acquisition platform including LiDAR, a camera, and a GNSS/INS on the agricultural machine. In terms of perception method, considering the different agricultural conditions, we used our datasets to train an effective 3D obstacle detector, known as Focal Voxel R-CNN. We used focal sparse convolution to replace the original 3D sparse convolution because of its adaptable ability to extract effective features from sparse point cloud data. Specifically, a branch of submanifold sparse convolution was added to the upstream of the backbone convolution network; this adds weight to the foreground point and retains more valuable information. In comparison with Voxel R-CNN, the proposed Focal Voxel R-CNN significantly improves the detection performance for small objects, and the AP in the pedestrian class increased from 89.04% to 92.89%. The results show that our model obtains an mAP of 91.43%, which is 3.36% higher than the base model. The detection speed is 28.57 FPS, which is 4.18 FPS faster than the base model. The experiments show the effectiveness of our model, which can provide a more reliable obstacle detection model for autonomous agricultural machines.

Within the transition from precision agriculture (task-specific approach) to smart farming (system-specific approach) there is a need to build and evaluate robotic systems that are part of an overall integrated system under a continuous two-way connection and interaction. This paper presented an initial step in creating an integrated system for agri-robotics, enabling two-way communication between an unmanned ground vehicle (UGV) and a farm management information system (FMIS) under the general scope of smart farming implementation. In this initial step, the primary task of route-planning for the agricultural vehicles, as a prerequisite for the execution of any field operation, was selected as a use-case for building and evaluating this integration. The system that was developed involves advanced route-planning algorithms within the cloud-based FMIS, a comprehensive algorithmic package compatible with agricultural vehicles utilizing the Robot Operating System (ROS), and a communicational and computational unit (CCU) interconnecting the FMIS algorithms, the corresponding user interface, and the vehicles. Its analytical module provides valuable information about UGVs’ performance metrics, specifically performance indicators of working distance, non-working distance, overlapped area, and field-traversing efficiency. The system was demonstrated via the implementation of two robotic vehicles in route-execution tasks in various operational configurations, field features, and cropping systems (open field, row crops, orchards). The case studies showed variability in the operational performance of the field traversal efficiency to be between 79.2% and 93%, while, when implementing the optimal route-planning functionality of the system, there was an improvement of up to 9.5% in the field efficiency. The demonstrated results indicate that the user can obtain better control over field operations by making alterations to ensure optimum field performance, and the user can have complete supervision of the operation.

In large-scale arable farming, multiple sequential operations involving multiple machines must be carried out simultaneously due to restrictions of short time windows. However, the coordination and planning of multiple sequential operations is a nontrivial task for farmers, since each operation may have its own set of operational features, e.g., operating width and turning radius. Taking the two sequential operations—hoeing cultivation and seeding—as an example, the seeder has double the width of the hoeing cultivator, and the seeder must remain idle while waiting for the hoeing cultivator to finish two rows before it can commence its seeding operation. A flow-shop working mode can coordinate multiple machines in multiple operations within a field when different operations have different implement widths. To this end, an auto-steering-based collaborative operating system for fleet management (FMCOS) was developed to realize an in-field flow-shop working mode, which is often adopted by the scaled agricultural machinery cooperatives. This paper proposes the structure and composition of the FMCOS, the method of operating strip segmenting, and a new algorithm for strip state updating between successive field operations under an optimal strategy for waiting time conditioning between sequential operations. A simulation model was developed to verify the state-updating algorithm. Then, the prototype system of FMCOS was combined with auto-steering systems on tractors, and the collaborative operating system for the server was integrated. Three field experiments of one operation, two operations, and three operations were carried out to verify the functionality and performance of FMCOS. The results of the experiment showed that the FMCOS could coordinate in-field fleet operations while improving both the job quality and the efficiency of fleet management by adopting the flow-shop working mode.

The classification that distinguishes whether machines are driving on roads or working in fields based on their global navigation satellite system (GNSS) trajectories is essential for effective management of cross-regional agricultural machinery services in China. In this paper, a novel field–road classification method utilizing multiple deep neural networks (MultiDNN) is proposed to enhance the accuracy of field and road point classification. The MultiDNN model incorporates a bi-directional long short-term memory network (BiLSTM), a topology adaptive graph convolution network (TAG), and a self-attention network (ATT) to effectively extract spatio-temporal features for field–road classification. The BiLSTM is used to capture temporal relationships along the time axis of a trajectory, providing global contextual information for each point. Then, the TAG network is used to obtain the spatio-temporal relationships between adjacent points in a trajectory, offering local contextual information for each point. Finally, the ATT network assigns varying weights to features to emphasize important characteristics. The performance of the MultiDNN model was evaluated using a wheat harvesting trajectory dataset, and the results showed that it achieved a high degree of accuracy, up to 89.75%, outperforming the best baseline method (GCN) by 2.79%.

Identifying the in-field trajectories of harvests is important for the activity analysis of agricultural machinery. This paper presents a K-means-based trajectory identification method that can automatically detect the “turning”, “working”, and “abnormal working” trajectories for wheat harvester in-field operation scenarios. This method contains two stages: clustering and correction. The clustering stage performs by the two-step K-means iterative clustering method (D-K-means). In the correction stage, the first step (M1) is performed based on the three distance features between the trajectory segments and the cluster center of the trajectory segments. The second step (M2) is based on the direction change of the “turning” and “abnormal working” trajectories. The third correction step (M3) is based on the operating characteristics to specify the start and stop positions of the turning. The developed method was validated by 50 trajectories. The results for the three trajectories and the five time intervals from 1 s to 5 s both have f1-scores above 0.90, and the f1-score using only the clustering method and the method of this paper increased from 0.55 to 0.95. After removing the turning and abnormal operation trajectories, the error of calculating farmland area with distance algorithm is reduced by 17.04% compared with that before processing.

DR-XGBoost: An XGBoost model for field-road segmentation based on dual feature extraction and recursive feature elimination