A prototype drone is presented that finds people in densely occluded forests fully autonomously.

Drones are becoming increasingly popular for remote sensing of landscapes in archeology, cultural heritage, forestry, and other disciplines. They are more efficient than airplanes for capturing small areas, of up to several hundred square meters. LiDAR (light detection and ranging) and photogrammetry have been applied together with drones to achieve 3D reconstruction. With airborne optical sectioning (AOS), we present a radically different approach that is based on an old idea: synthetic aperture imaging. Rather than measuring, computing, and rendering 3D point clouds or triangulated 3D meshes, we apply image-based rendering for 3D visualization. In contrast to photogrammetry, AOS does not suffer from inaccurate correspondence matches and long processing times. It is cheaper than LiDAR, delivers surface color information, and has the potential to achieve high sampling resolutions. AOS samples the optical signal of wide synthetic apertures (30–100 m diameter) with unstructured video images recorded from a low-cost camera drone to support optical sectioning by image integration. The wide aperture signal results in a shallow depth of field and consequently in a strong blur of out-of-focus occluders, while images of points in focus remain clearly visible. Shifting focus computationally towards the ground allows optical slicing through dense occluder structures (such as leaves, tree branches, and coniferous trees), and discovery and inspection of concealed artifacts on the surface.

Synthetic apertures find applications in many fields, such as radar, radio telescopes, microscopy, sonar, ultrasound, LiDAR, and optical imaging. They approximate the signal of a single hypothetical wide aperture sensor with either an array of static small aperture sensors or a single moving small aperture sensor. Common sense in synthetic aperture sampling is that a dense sampling pattern within a wide aperture is required to reconstruct a clear signal. In this article we show that there exists practical limits to both, synthetic aperture size and number of samples for the application of occlusion removal. This leads to an understanding on how to design synthetic aperture sampling patterns and sensors in a most optimal and practically efficient way. We apply our findings to airborne optical sectioning which uses camera drones and synthetic aperture imaging to computationally remove occluding vegetation or trees for inspecting ground surfaces.

We apply a multi-spectral (RGB and thermal) camera drone for synthetic aperture imaging to computationally remove occluding vegetation for revealing hidden objects, as required in archeology, search-and-rescue, animal inspection, and border control applications. The radiated heat signal of strongly occluded targets, such as a human bodies hidden in dense shrub, can be made visible by integrating multiple thermal recordings from slightly different perspectives, while being entirely invisible in RGB recordings or unidentifiable in single thermal images. We collect bits of heat radiation through the occluder volume over a wide synthetic aperture range and computationally combine them to a clear image. This requires precise estimation of the drone’s position and orientation for each capturing pose, which is supported by applying computer vision algorithms on the high resolution RGB images.

Synthetic apertures sample the signal of wide aperture sensors with either arrays of static or single moving smaller aperture sensors whose individual signals are computationally combined to increase the resolution, depth-of-field, frame rate, contrast, and signal-to-noise ratio. This principle has been used for radar, telescopes, microscopes, sonar, ultrasound, laser, and optical imaging. With airborne optical sectioning (AOS), we apply camera drones for synthetic aperture imaging to uncover the ruins of a 19th century fortification system that is concealed by dense forest and shrubs. Compared to alternative airborne scanning technologies (such as LiDAR), AOS is cheaper, delivers surface color information, achieves higher sampling resolutions, and (in contrast to photogrammetry) does not suffer from inaccurate correspondence matches and long processing times.

In this letter, we describe and validate the first fully automatic parameter optimization for thermal synthetic aperture visualization. It replaces previous manual exploration of the parameter space, which is time-consuming and error-prone. We prove that the visibility of targets in thermal integral images is proportional to the variance of the targets' image. Since this is invariant to occlusion, it represents a suitable objective function for optimization. Our findings have the potential to enable fully autonomous search and recuse operations with camera drones.