Given the large volume of remote sensing images collected daily, automatic object detection and segmentation have been a consistent need in Earth observation (EO). However, objects of interest vary in shape, size, appearance, and reflecting properties. This is not only reflected by the fact that these objects exhibit differences due to their geographical diversity but also by the fact that these objects appear differently in images collected from different sensors (optical and radar) and platforms (satellite, aerial, and unmanned aerial vehicles (UAV)). Although there exists a plethora of object detection methods in the area of remote sensing, given the very fast development of prevalent deep learning methods, there is still a lack of recent updates for object detection methods. In this paper, we aim to provide an update that informs researchers about the recent development of object detection methods and their close sibling in the deep learning era, instance segmentation. The integration of these methods will cover approaches to data at different scales and modalities, such as optical, synthetic aperture radar (SAR) images, and digital surface models (DSM). Specific emphasis will be placed on approaches addressing data and label limitations in this deep learning era. Further, we survey examples of remote sensing applications that benefited from automatic object detection and discuss future trends of the automatic object detection in EO.

Hyperspectral imaging is capable of capturing information beyond conventional RGB cameras; therefore, several applications of this have been found, such as material identification and spectral analysis. However, similar to many camera systems, most of the existing hyperspectral cameras are still passive imaging systems. Such systems require an external light source to illuminate the objects, to capture the spectral intensity. As a result, the collected images highly depend on the environment lighting and the imaging system cannot function in a dark or low-light environment. This work develops a prototype system for active hyperspectral imaging, which actively emits diverse single-wavelength light rays at a specific frequency when imaging. This concept has several advantages: first, using the controlled lighting, the magnitude of the individual bands is more standardized to extract reflectance information; second, the system is capable of focusing on the desired spectral range by adjusting the number and type of LEDs; third, an active system could be mechanically easier to manufacture, since it does not require complex band filters as used in passive systems. Three lab experiments show that such a design is feasible and could yield informative hyperspectral images in low light or dark environments: (1) spectral analysis: this system’s hyperspectral images improve food ripening and stone type discernibility over RGB images; (2) interpretability: this system’s hyperspectral images improve machine learning accuracy. Therefore, it can potentially benefit the academic and industry segments, such as geochemistry, earth science, subsurface energy, and mining.

Across the planet, Indigenous societies control, use, and manage large tracts of tropical forest that are crucial for combating climate change. Here we investigate whether customary Indigenous agricultural practices can increase forest species diversity using swidden (aka, slash-and-burn) agriculture. We examine the community lands surrounding two Q’eqchi’ Maya villages in southern Belize using high-resolution multispectral imagery that we collected with drones, ground truth data related to land use and history, and a plant and tree biodiversity survey. We use computational methods including simulation and Bayesian modeling to analyze the relationship between spectral diversity, forest diversity, and landscape disturbance. Our key result, replicated in both villages, is that the highest level of spectral diversity (a proxy for forest diversity) is associated with an intermediate level of forest disturbance. In conclusion, we suggest that concepts from complex adaptive systems can provide an important conceptual framework for understanding how ecosystem enhancement can emerge from customary Indigenous land use practices.

Quantifying the colors of objects is useful in a wide range of applications, including medical diagnosis, agricultural monitoring, and food safety. Accurate colorimetric measurement of objects is a laborious process normally performed through a color matching test in the laboratory. A promising alternative is to use digital images for colorimetric measurement, due to their portability and ease of use. However, image-based measurements suffer from errors caused by the non-linear image formation process and unpredictable environmental lighting. Solutions to this problem often perform relative color correction among multiple images through discrete color reference boards, which may yield biased results due to the lack of continuous observation. In this paper, we propose a smartphone-based solution, that couples a designated color reference board with a novel color correction algorithm, to achieve accurate and absolute color measurements. Our color reference board contains multiple color stripes with continuous color sampling at the sides. A novel correction algorithm is proposed to utilize a first-order spatial varying regression model to perform the color correction, which leverages both the absolute color magnitude and scale to maximize the correction accuracy. The proposed algorithm is implemented as a “human-in-the-loop” smartphone application, where users are guided by an augmented reality scheme with a marker tracking module to take images at an angle that minimizes the impact of non-Lambertian reflectance. Our experimental results show that our colorimetric measurement is device independent and can reduce up to 90% color variance for images collected under different lighting conditions. In the application of reading pH values from test papers, we show that our system performs 200% better than human reading. The designed color reference board, the correction algorithm, and our augmented reality guiding approach form an integrated system as a novel solution to measure color with increased accuracy. This technique has the flexibility to improve color reading performance in systems beyond existing applications, evidenced by both qualitative and quantitative experiments on example applications such as pH-test reading.

Deep learning has significantly advanced building segmentation in remote sensing, yet models struggle to generalize on data of diverse geographic regions due to variations in city layouts and the distribution of building types, sizes and locations. However, the amount of time-consuming annotated data for capturing worldwide diversity may never catch up with the demands of increasingly data-hungry models. Thus, we propose a novel approach: re-training models at test time using synthetic data tailored to the target region’s city layout. This method generates geo-typical synthetic data that closely replicates the urban structure of a target area by leveraging geospatial data such as street network from OpenStreetMap. Using procedural modeling and physics-based rendering, very high-resolution synthetic images are created, incorporating domain randomization in building shapes, materials, and environmental illumination. This enables the generation of virtually unlimited training samples that maintain the essential characteristics of the target environment. To overcome synthetic-to-real domain gaps, our approach integrates geo-typical data into an adversarial domain adaptation framework for building segmentation. Experiments demonstrate significant performance enhancements, with median improvements of up to 12%, depending on the domain gap. This scalable and cost-effective method blends partial geographic knowledge with synthetic imagery, providing a promising solution to the “model collapse” issue in purely synthetic datasets. It offers a practical pathway to improving generalization in remote sensing building segmentation without extensive real-world annotations. https://github.com/GDAOSU/geotypical_synthetic_label_building_detection.