Recent advancements in deep learning techniques have spurred considerable interest in their application to hyperspectral imagery processing. This paper provides a comprehensive review of the latest developments in this field, focusing on methodologies, challenges, and emerging trends. Deep learning architectures such as Convolutional Neural Networks (CNNs), autoencoders, Deep Belief Networks (DBNs), Generative Adversarial Networks (GANs), and Recurrent Neural Networks (RNNs) are examined for their suitability in processing hyperspectral data. Key challenges, including limited training data and computational constraints, are identified, along with strategies such as data augmentation and noise reduction using GANs. This paper discusses the efficacy of different network architectures, highlighting the advantages of lightweight CNN models and 1D-CNNs for onboard processing. Moreover, the potential of hardware accelerators, particularly Field Programmable Gate Arrays (FPGAs), for enhancing processing efficiency is explored. This review concludes with insights into ongoing research trends, including the integration of deep learning techniques into Earth observation missions such as the CHIME mission, and emphasizes the need for further exploration and refinement of deep learning methodologies to address the evolving demands of hyperspectral image processing.
7 November 2024 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Satellites are increasingly adopting onboard AI to optimize operations and increase autonomy through in-orbit inference. The use of deep learning (DL) models for segmentation in hyperspectral imagery offers advantages for remote sensing applications. In this work, we train and test 20 models for multiclass segmentation in hyperspectral imagery, selected for their potential in future space deployment. These models include 1-D and 2-D convolutional neural networks (CNNs) and the latest vision transformers (ViTs). We propose a lightweight 1D-CNN model, 1D-Justo-LiuNet, which outperforms state-of-the-art models in the hypespectral domain. 1D-Justo-LiuNet exceeds the performance of 2D-CNN UNets and outperforms Apple's lightweight vision transformers designed for mobile inference. 1D-Justo-LiuNet achieves the highest accuracy (0.93) with the smallest model size (4563 parameters) among all tested models, while maintaining fast inference. Unlike 2D-CNNs and ViTs, which encode both spectral and spatial information, 1D-Justo-LiuNet focuses solely on the rich spectral features in hyperspectral data, benefitting from the high-dimensional feature space. Our findings are validated across various satellite datasets, with the HYPSO-1 mission serving as the primary case study for sea, land, and cloud segmentation. We further confirm our conclusions through generalization tests on other hyperspectral missions, such as NASA's EO-1. Based on its superior performance and compact size, we conclude that 1D-Justo-LiuNet is highly suitable for in-orbit deployment, providing an effective solution for optimizing and automating satellite operations at edge.
AI on spaceborne platforms optimizes operations and increases automation, crucial for satellites with limited downlink capacity. It can ensure that only valuable information is transmitted, minimizing resources spent on unnecessary data, which is especially important in hyperspectral Earth Observation missions, producing large data volumes. Our previous work showed that the 1D-CNN, 1D-Justo-LiuNet, outperformed 2D-CNNs and Vision Transformers for hyperspectral segmentation with an accuracy of 0.93 and 4563 parameters, making our model the best choice for in-orbit deployment. While the state of the art has deployed 1D-CNNs on low-power platforms, such as Unmanned Aerial Vehicles, they have still not been deployed in space before. In this work, we mark the first deployment and testing of a 1D-CNN in a satellite. We implement a C version of the 1D-Justo-LiuNet and, after ground validation, we deploy it on board the HYPSO-1 satellite. We demonstrate in-flight segmentation of hyperspectral images via the 1D-CNN to classify pixels into sea, land, and cloud categories. We show how in-orbit segmentation improves satellite operations, increases automation, and optimizes downlink. We give examples of how in-orbit segmentation addresses mission challenges in HYPSO-1, such as incomplete data reception, incorrect satellite pointing, and cloud cover, helping to decide whether to transmit or discard data on board. An additional CNN autonomously interprets the segmented images, enabling on-board decisions on data downlink.
AI on spaceborne platforms optimizes operations and increases automation, crucial for satellites with limited downlink capacity. It can ensure that only valuable information is transmitted, minimizing resources spent on unnecessary data, which is especially important in hyperspectral Earth Observation missions, producing large data volumes. Our previous work showed that the 1D-CNN, 1D-Justo-LiuNet, outperformed 2D-CNNs and Vision Transformers for hyperspectral segmentation, making our model the best choice for in-orbit deployment. In this work, we mark the first deployment and testing of a 1D-CNN in a satellite. We implement a C version of the 1D-Justo-LiuNet and, after ground validation, we deploy it on the on board the HYPSO-1 satellite. We demonstrate in-flight segmentation of hyperspectral images via the 1D-CNN to classify pixels into sea, land, and cloud categories. We show how in-orbit segmentation improves satellite operations, increases automation, and optimizes downlink. We give examples of how in-orbit segmentation addresses mission challenges in HYPSO-1, such as incomplete data reception, incorrect satellite pointing, and cloud cover, helping decide whether to transmit or discard data on board. An additional CNN autonomously interprets segmented images, enabling on-board decisions on data downlink.