Automated robotic platforms are an important part of precision agriculture solutions for sustainable food production. Agri-robots require robust and accurate guidance systems in order to navigate between crops and to and from their base station. Onboard sensors such as machine vision cameras offer a flexible guidance alternative to more expensive solutions for structured environments such as scanning lidar or RTK-GNSS. The main challenges for visual crop row guidance are the dramatic differences in appearance of crops between farms and throughout the season and the variations in crop spacing and contours of the crop rows. Here we present a visual guidance pipeline for an agri-robot operating in strawberry fields in Norway that is based on semantic segmentation with a convolution neural network (CNN) to segment input RGB images into crop and not-crop (i.e., drivable terrain) regions. To handle the uneven contours of crop rows in Norway’s hilly agricultural regions, we develop a new adaptive multi-ROI method for fitting trajectories to the drivable regions. We test our approach in open-loop trials with a real agri-robot operating in the field and show that our approach compares favourably to other traditional guidance approaches.

For most fruit-harvesting robots, an essential task of the machine vision system is to provide the manipulator with an accurate three-dimensional location of the target. However, the accuracy of this location can be affected by various factors. This study aimed to develop seven location methods, to investigate their effectiveness, as well as the influences of different camera modes and camera types, and, ultimately, to ascertain which was the optimal method. These methods utilized the pixels of the detected targets in each image, the corresponding depth values, as well as the locations of the 2D bounding boxes extracted from the detection results. These location methods differed in the way that they obtained the position of the 3D bounding box, and in their use of point clustering or colour thresholding. The images were collected via two types of 3D camera, patterned structured light and time-of-flight. Comparative analysis showed that methods using the 2D bounding box and the selected depth value to calculate the 3D bounding box were faster (0.2–8.4 ms compared to 151.9–325.2 ms) and performed better than the 3D clustering methods. In addition, four modes of the structured light camera were tested and compared. The results showed that the high-accuracy mode had fewer noise points but a lower location rate (89.2–89.9%), while the high-density mode created more noise points but a higher location rate (98.9%). Evaluations also indicated that the data from the time-of-flight camera better represented the 3D shape (26.3% more accurate along the camera’s depth direction). Therefore, time-of-flight camera was considered better for the applications that required more accurate 3D shape. This paper, thus, provided references in the selection of location methods, cameras and corresponding modes for related work.

This paper presents a pig carcass cutting dataset, captured from a bespoke frame structure with 6 Intel® RealSense™ Depth Camera D415 cameras attached, and later recorded from a single camera attached to a robotic arm cycling through the positions previously defined by the frame structure. The data is composed of bags files recorded from the Intel’s SDK, which includes RGB-D data and camera intrinsic parameters for each sensor. In addition, ten JSON files with the transformation matrix for each camera in relation to the left/front camera in the structure are provided, five JSON files for the data recorded with the bespoke frame and five JSON files for the data captured with the robotic arm.

Advances in visual sensor devices and computing power are revolutionising the interaction of robots with their environment. Cameras that capture depth information along with a common colour image play a significant role. These devices are cheap, small, and fairly precise. The information provided, particularly point clouds, can be generated in a virtual computing environment, providing complete 3D information for applications. However, off-the-shelf cameras often have a limited field of view, both on the horizontal and vertical axis. In larger environments, it is therefore often necessary to combine information from several cameras or positions. To concatenate multiple point clouds and generate the complete environment information, the pose of each camera must be known in the outer scene, i.e., they must reference a common coordinate system. To achieve this, a coordinate system must be defined, and then every device must be positioned according to this coordinate system. For cameras, a calibration can be performed to find its pose in relation to this coordinate system. Several calibration methods have been proposed to solve this challenge, ranging from structured objects such as chessboards to features in the environment. In this study, we investigate how three different pose estimation methods for multi-camera perspectives perform when reconstructing a scene in 3D. We evaluate the usage of a charuco cube, a double-sided charuco board, and a robot’s tool centre point (TCP) position in a real usage case, where precision is a key point for the system. We define a methodology to identify the points in the 3D space and measure the root-mean-square error (RMSE) based on the Euclidean distance of the actual point to a generated ground-truth point. The reconstruction carried out using the robot’s TCP position produced the best result, followed by the charuco cuboid; the double-sided angled charuco board exhibited the worst performance.

Safe navigation is a key objective for autonomous applications, particularly those involving mobile tasks, to avoid dangerous situations and prevent harm to humans. However, the integration of a risk management process is not yet mandatory in robotics development. Ensuring safety using mobile robots is critical for many real-world applications, especially those in which contact with the robot could result in fatal consequences, such as agricultural environments where a mobile device with an industrial cutter is used for grass-mowing. In this paper, we propose an explicit integration of a risk management process into the design of the software for an autonomous grass mower, with the aim of enhancing safety. Our approach is tested and validated in simulated scenarios that assess the effectiveness of different custom safety functionalities in terms of collision prevention, execution time, and the number of required human interventions.