This article proposes a novel approach to segment instances of bulk material heaps in aerial data using deep learning-based computer vision and transfer learning to automate material inventory procedures in the construction-, mining-, and material-handling industry. The proposed method uses information about color, texture, and surface topography as input features for a supervised computer vision algorithm. The approach neither relies on hand-crafted assumptions on the general shape of heaps, nor does it solely rely on surface material type recognition. Therefore, the method is able to (1) segment heaps with “atypical” shapes, (2) segment heaps that stand on a surface made of the same material as the heap itself, (3) segment individual heaps of the same material type that border each other, and (4) differentiate between artificial heaps and other objects of similar shape like natural hills. To utilize well-established segmentation algorithms for raster-grid-based data structures, this study proposes a pre-processing step to remove all overhanging occlusions from a 3D surface scan and convert it into a 2.5D raster format. Preliminary results demonstrate the general feasibility of the approach. The average F1 score computed on the test set was 0.70 regarding object detection and 0.90 regarding the pixelwise segmentation.

Asphalt road construction projects pose a lot of challenges in different disciplines, from the choice of construction methods, type and quantity of machines and personnel, pre-planning of all processes, ordering material from asphalt mixing plants, real-time logistics of handling material transport, and the asphalt paving and compaction process itself. Depending on the complexity and size of the construction site, preparation and execution can be influenced by a lot of unknown variables, like weather impact, material properties of the delivered asphalt or delays in the deliveries themselves. These circumstances might lead to irregularities in the asphalt material temperatures and thus suboptimal conditions for compaction or even construction stoppage - in any case, they lead to a reduction in quality of the built asphalt road. In recent years, a lot of mobile devices, sensors and software have been emerging as viable solutions in the construction industry to digitize and support these processes. Those real-time applications allow deeper insight into logistics or material properties on site and acquire data for downstream analysis. These endeavours in process optimization now benefit construction companies to increase efficiency, and for work safety and environmental aspects, it is crucial to optimize energy and CO2 consumption. Improved logistics and building roads with low-temperature asphalt will have to become a standard in road construction. The present study gives an overview of the digitalized processes in asphalt road construction with market-available solutions focusing on logistics data of the material delivery chain from mixing plant to paver, thermal imaging data of the built asphalt to validate asphalt temperatures, as well as machine and location data from asphalt paver and compaction rollers.

This paper proposes a method to recognize the current progress of a road construction site by combining geometric comparison of 3D CAD geometry to as-built point cloud data with image-based surface recognition. Recognizing the current status of road construction sites by purely geometric means is error-prone, since the individual road layers are relatively thin compared to possible deviations of the built road from the design geometry and possible inaccuracies in the z-dimension of the as-built point cloud. Recognizing the current state by purely visual means is prone to error if two or more layers look alike or if the appearance of a layer is distorted by e.g. dust. Therefore, we propose to merge the geometric and the visual evaluation into one hybrid probabilistic algorithm. The goal is not only to improve the accuracy of the progress recognition, but also to gain additional information about the confidence of the systems decision.

Modern construction machines are equipped with state-of-the-art sensors, like accurate GNSS receivers, thermal cameras or distance and vibration sensors, that record large volumes of data on their spatial and temporal properties, construction methods or the structural properties of the built material. While it is a challenge itself to condense insights from a single data source, the task grows exponentially more complex when several machines—and, with that, data sources—are involved. The present study covers the merging of data on different road and asphalt construction machines, collected on hundreds of construction sites for quality analysis, and describes relevant approaches in data engineering necessary to infer insights across the entire construction process. This publication proposes an approach to analyzing both individual metrics and spatially merged multi-variables. Depending on the step within the construction process, or the personnel involved, it can be important to perform actions based on detailed multi-variable information, or, when a timely decision is necessary, based on real-time high-level information. To enable this, data engineering methods and a proposed system have been developed, and concrete, actionable results from the data are presented. A representative sample project includes data covering a 14 h construction span with 62 material deliveries, thousands of temperature readings of the newly built asphalt (0.25 m × 0.25 m temperature grid at about 130 ± 15 °C), paver control (mostly steady speed at 3.55 m/min) and recorded track data on asphalt rollers, which are condensed into resulting rollovers.

Support structures of relocatable large-scale outdoor conveyor systems are prone to unintended movements over time due to vibration induced by their moving parts, as well as the usually non-solid foundation ground. These movements can lead to a misalignment of the system’s individual segments, which in turn can cause an increase in power consumption as well as a potential belt ripping and therefore a breakdown of the system. This article proposes a novel approach to autonomously monitor the alignment of large-scale conveyor systems utilizing aerial images captured via unmanned aerial vehicles, semantic image segmentation via deep learning, 2D-to-3D label transfer using photogrammetry, and 3D point cloud analysis. The feasibility of the method is demonstrated on a working prototype. One of the main challenges is to find a good balance between precision and reliability on the one hand and computational costs on the other.