With advances in precision agriculture, autonomous agricultural machines can reduce human labor, optimize workflow, and increase productivity. Accurate and reliable obstacle-detection and avoidance systems are essential for ensuring the safety of automated agricultural machines. Existing LiDAR-based obstacle detection methods for the farmland environment process the point clouds via manually designed features, which is time-consuming, labor-intensive, and weak in terms of generalization. In contrast, deep learning has a powerful ability to learn features autonomously. In this study, we attempted to apply deep learning in LiDAR-based 3D obstacle detection for the farmland environment. In terms of perception hardware, we established a data acquisition platform including LiDAR, a camera, and a GNSS/INS on the agricultural machine. In terms of perception method, considering the different agricultural conditions, we used our datasets to train an effective 3D obstacle detector, known as Focal Voxel R-CNN. We used focal sparse convolution to replace the original 3D sparse convolution because of its adaptable ability to extract effective features from sparse point cloud data. Specifically, a branch of submanifold sparse convolution was added to the upstream of the backbone convolution network; this adds weight to the foreground point and retains more valuable information. In comparison with Voxel R-CNN, the proposed Focal Voxel R-CNN significantly improves the detection performance for small objects, and the AP in the pedestrian class increased from 89.04% to 92.89%. The results show that our model obtains an mAP of 91.43%, which is 3.36% higher than the base model. The detection speed is 28.57 FPS, which is 4.18 FPS faster than the base model. The experiments show the effectiveness of our model, which can provide a more reliable obstacle detection model for autonomous agricultural machines.

Capacitated field operations involve input/output material flows where there are capacity constraints in the form of a specific load that a vehicle can carry. As such, a specific normal-sized field cannot be covered in one single operation using only one load, and the vehicle needs to get serviced (i.e., refilling) from out-of-field facilities (depot). Although several algorithms have been developed to solve the routing problem of capacitated operations, these algorithms only considered one depot. The general goal of this paper is to develop a route planning tool for agricultural machines with multiple depots. The tool presented consists of two modules: the first one regards the field geometrical representation in which the field is partitioned into tracks and headland passes; the second one regards route optimization that is implemented by the metaheuristic simulated annealing (SA) algorithm. In order to validate the developed tool, a comparison between a well-known route planning approach, namely B-pattern, and the algorithm presented in this study was carried out. The results show that the proposed algorithm outperforms the B-pattern by up to 20.0% in terms of traveled nonworking distance. The applicability of the tool developed was tested in a case study with seven scenarios differing in terms of locations and number of depots. The results of this study illustrated that the location and number of depots significantly affect the total nonworking traversal distance during a field operation.

The classification that distinguishes whether machines are driving on roads or working in fields based on their global navigation satellite system (GNSS) trajectories is essential for effective management of cross-regional agricultural machinery services in China. In this paper, a novel field–road classification method utilizing multiple deep neural networks (MultiDNN) is proposed to enhance the accuracy of field and road point classification. The MultiDNN model incorporates a bi-directional long short-term memory network (BiLSTM), a topology adaptive graph convolution network (TAG), and a self-attention network (ATT) to effectively extract spatio-temporal features for field–road classification. The BiLSTM is used to capture temporal relationships along the time axis of a trajectory, providing global contextual information for each point. Then, the TAG network is used to obtain the spatio-temporal relationships between adjacent points in a trajectory, offering local contextual information for each point. Finally, the ATT network assigns varying weights to features to emphasize important characteristics. The performance of the MultiDNN model was evaluated using a wheat harvesting trajectory dataset, and the results showed that it achieved a high degree of accuracy, up to 89.75%, outperforming the best baseline method (GCN) by 2.79%.

Digital twins in smart farming: An autoware-based simulator for autonomous agricultural vehicles

Advanced systems for manned and/or agricultural vehicles—such as systems for auto-steering, navigation-adding, and autonomous route planning—require new capabilities in terms of the internal representation for the autonomous system of the working space; that is, the generation of a metric map that provides by numerical parameters any operation-related entity of the working space. In this paper, a real-time approach was developed for the generation of the field metric map, based on a row generation method (polygons-based geometry). The approach can deal with fields with or without in-field obstacles, where the generated field-work tracks can be either straight or curved. The functionality of the approach was demonstrated on 12 fields with different number of obstacles ranging from one to six. The test results showed that the computational times were in the range of 0.26–24.51 s. The presented tool brings a number of advancements on the process of generating a metric map for arable farming field operations, including the real-time generation feature, the potential to deal with multiple-obstacle areas, and the reduction in the overlapped area.

Precision Farming (PF) and Controlled Traffic Farming (CTF) are well known concepts within agriculture, but the adoption rate of these practices by farmers is still very low, because farmers lack the needed skills or fail to see the benefits of using these practices. If farmers want to reap the full benefits, operational planning must be carried out in advance for the entire crop cycle, before the crop season begins. However, operational planning across the entire crop cycle is a non-trivial task, since the efficiency of each operation is determined by a range of selected operational features (e.g., wayline direction, operational speed, vehicle capacity, wayline sequence, and turn type). To that end, we present, in this paper, an application that can support farmers with operational planning of field operations with CTF, by automating the process. It provides the farmer with an overview of all his field operations, and acts as a decision support tool during the operational planning process. The application allows farmers to store and manage field and equipment information, which is used as input, when setting up CTF and generating way lines and route plans for the individual fields. One of the key benefits of the application is the provided comparison feature, where farmers can compare alternative solutions, based on Key Performance Indicators (KPIs). Results from an example field, for operations with different machine setups, are presented to illustrate how KPIs and visualisations can support farmers during the decision process.