We investigate the problem of colourising a point cloud using an arbitrary mobile mapping device and an independent camera. The loose coupling of sensors creates a number of challenges related to timing, point visibility and colour determination. We address each of these problems and demonstrate our resulting algorithm on data captured using sensor payloads mounted to hand-held, aerial and ground systems, illustrating the `plug-and-play' portability of the method.

The evolution of orchard production systems towards higher density layouts, makes monitoring of canopy and disease increasingly important. Technological advances over the last few years have greatly increased our ability to collect, collate and analyse our data on a per-tree basis at large orchard scales. We call this the Digital-Twin Orchard. A digital-twin is a virtual model of every tree and surroundings. The pairing of the virtual and physical worlds allows analysis of data and continuous monitoring of orchards production systems to predict stress, disease and crop losses, and to develop new opportunities for end-to-end learning. Monitoring of orchards is not a new concept but the digital-twin is a continuously learning system that could be queried automatically to analyse specific outcomes under varying simulated environmental and orchard management parameters. Digital-twin enables improvement of production and dynamic prediction of disease, stress and yield gaps using an end-to-end AI platform. In this paper, we present AgScan3D+: our automated dynamic canopy monitoring system to generate a digital-twin of every tree on a large orchard scale. AgScan3D+ consists of a spinning 3D LiDAR plus cameras that can be retrofitted to a farm vehicle and provides real time on-farm decision support by monitoring the condition of every plant in 3D such as their health, structure, stress, fruit quality, and more. The proposed system has been trialled in mango, macadamia, avocado and grapevines orchards and generated a digital-twin of 15,000 trees. The results were used to model canopy structural characteristics such as foliage density and light penetration distribution.

We present a novel method for mapping general three-dimensional environments, where sufficient geometric or visual information is not everywhere guaranteed and where the device motion is unconstrained as with handheld systems. The continuous-time simultaneous localization and mapping algorithm integrates a lidar, camera, and inertial measurement unit in a complementary fashion whereby all sensors contribute constraints to the optimization. The proposed algorithm is designed to expand the domain of mappable environments and therefore increase the reliability and utility of general purpose mobile mapping. A key component of the proposed algorithm is the incorporation of depth uncertainty into visual features, which is effective for noisy surfaces and allows features with and without depth estimates to be modeled in a unified manner. Results demonstrate a wider mappable domain on challenging environments compared to the state-of-the-art lidar or vision-based localization and mapping algorithms.

Generating accurate digital tree models from scanned environments is invaluable for forestry, agriculture, and other outdoor industries in tasks such as identifying fall hazards, estimating trees’ biomass and calculating traversability. Existing methods for tree reconstruction rely on sparse feature identification to segment a forest into individual trees and generate a branch structure graph, limiting their application to easily separable trees and uniform forests. However, the natural world is a messy place in which trees present with significant heterogeneity and are frequently encroached upon by the surrounding environment. We present a general method for extracting the branch structure of trees from point cloud data, which estimates the structure of trees by adapting the methods of structural topology optimisation to find the optimal material distribution to interpolate the input data. We present the results of this optimisation over a wide variety of scans, and discuss the benefits and drawbacks of this novel approach to tree structure reconstruction. Our method generates detailed and accurate tree structures, with a mean Surface Error (SE) of 15 cm over 13 diverse tree datasets.