In recent years, massive digitisation of cultural heritage (CH) assets has become a focus of European programmes and initiatives. Among CH settings, attention is reserved to the immense and precious museum collections, whose digital 3D reproduction can support broader non-invasive analyses and stimulate the realisation of more attractive and interactive exhibitions. The reconstruction pipeline typically includes numerous processing steps when passive techniques are selected to deal with object digitisation. This article presents some insights on critical operations, which, based on our experience, can rule the quality of the final models and the reconstruction times for delivering 3D heritage results, while boosting the sustainability of digital cultural contents. The depth of field (DoF) problem is explored in the acquisition phase when surveying medium and small-sized objects. Techniques for deblurring images and masking object backgrounds are examined relative to the pre-processing stage. Some point cloud denoising and mesh simplification procedures are analysed in data post-processing. Hints on physically-based rendering (PBR) materials are also presented as closing operations of the reconstruction pipeline. This paper explores these processes mainly through experiments, providing a practical guide, tricks, and suggestions when tackling museum digitisation projects.
10 March 2023 in ISPRS Journal of Photogrammetry and Remote Sensing
Historical images provide a valuable source of information exploited by several kinds of applications, such as the monitoring of cities and territories, the reconstruction of destroyed buildings, and are increasingly being shared for cultural promotion projects through virtual reality or augmented reality applications. Finding reliable and accurate matches between historical and present images is a fundamental step for such tasks since they require to co-register the present 3D scene with the past one. Classical image matching solutions are sensitive to strong radiometric variations within the images, which are particularly relevant in these multi-temporal contexts due to different types of sensitive media (film/sensors) employed for the image acquisitions, different lighting conditions and viewpoint angles. In this work, we investigate the actual improvement provided by recent deep learning approaches to match historical and nowadays images. As learning-based methods have been trained to find reliable matches in challenging scenarios, including large viewpoint and illumination changes, they could overcome the limitations of classic hand-crafted methods such as SIFT and ORB. The most relevant approaches proposed by the research community in the last years are analyzed and compared using pairs of multi-temporal images.
14 February 2024 in The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Finding corresponding points between images is a fundamental step in photogrammetry and computer vision tasks. Traditionally, image matching has relied on hand-crafted algorithms such as SIFT or ORB. However, these algorithms face challenges when dealing with multi-temporal images, varying radiometry and contents as well as significant viewpoint differences. Recently, the computer vision community has proposed several deep learning-based approaches that are trained for challenging illumination and wide viewing angle scenarios. However, they suffer from certain limitations, such as rotations, and they are not applicable to high resolution images due to computational constraints. In addition, they are not widely used by the photogrammetric community due to limited integration with standard photogrammetric software packages. To overcome these challenges, this paper introduces Deep-Image-Matching , an open-source toolbox designed to match images using different matching strategies, ranging from traditional hand-crafted to deep-learning methods (https://github.com/3DOM-FBK/deep-image-matching). The toolbox accommodates high-resolution datasets, e.g. data acquired with full-frame or aerial sensors, and addresses known rotation-related problems of the learned features. The toolbox provides image correspondences outcomes that are directly compatible with commercial and open-source software packages, such as COLMAP and openMVG, for a bundle adjustment. The paper includes also a series of cultural heritage case studies that present challenging conditions where traditional hand-crafted approaches typically fail.
30 May 2022 in The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Automatic photogrammetric processing of historical (or archival) aerial photos is still a challenging task, particularly in cases of missing ancillary information, low radiometric and image quality, limited stereo coverage or large temporal span. However, with recent advances in photogrammetry and Artificial Intelligence (AI) algorithms for image processing and interpretation, an increasing number of applications are now feasible. The article presents the TIME (hisTorical aerIal iMagEs) benchmark (https://time.fbk.eu/), promoted by EuroSDR to explore the potential of historical aerial images. Realized in collaboration with various European NMCAs, the benchmark has garnered aerial image blocks and time series imagery captured since the 1950s. To support the photogrammetric processing of the digitized photos, ancillary data are supplied with available information about flight missions, taking cameras, and ground control points (GCPs). Several diverse investigations have been undertaken with the benchmark datasets, all captured over historical urban areas or landscapes. The paper describes the benchmark datasets and some potential research topics, presenting several tests and analyses realized with the collated and shared data.
30 May 2022 in The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Aerial triangulation (AT) has reached outstanding progress in the last decades, and now fully automated solutions for nadir and oblique images are available. Usually, image correspondences (tie points) are found using hand-crafted methods, such as SIFT or its variants. But in the last years, there were many investigations and developments to promote the use of machine and deep learning solutions within the photogrammetric processing pipeline. The paper explores learning-based methods for the extraction of tie points in aerial image blocks. Image correspondences are used to perform aerial triangulation (AT) and successively generate dense point clouds. Two different datasets are used to compare conventional hand-crafted detector/descriptor methods with respect to learning-based methods. Accuracy analyses are performed using GCPs as well as ground truth LiDAR point clouds. Results confirm the potential of learningbased methods in finding reliable image correspondences in the aerial block, still showing space for improvements due to camera rotations.