With rapid advancements in remote sensing image registration algorithms, comprehensive imaging applications are no longer limited to single-modal remote sensing images. Instead, multi-modal remote sensing (MMRS) image registration has become a research focus in recent years. However, considering multi-source, multi-temporal, and multi-spectrum input introduces significant nonlinear radiation differences in MMRS images for which researchers need to develop novel solutions. At present, comprehensive reviews and analyses of MMRS image registration methods are inadequate in related fields. Thus, this paper introduces three theoretical frameworks: namely, area-based, feature-based and deep learning-based methods. We present a brief review of traditional methods and focus on more advanced methods for MMRS image registration proposed in recent years. Our review or comprehensive analysis is intended to provide researchers in related fields with advanced understanding to achieve further breakthroughs and innovations.

Over the past decade, using Interferometric Synthetic Aperture Radar (InSAR) remote sensing technology for ground displacement detection has become very successful. However, during the acquisition stage, microwave signals reflected from the ground and received by the satellite are contaminated, for example, due to undesirable material reflectance and atmospheric factors, and there is no clean ground truth to discriminate these noises, which adversely affect InSAR phase computation. Accurate InSAR phase filtering and coherence estimation are crucial for subsequent processing steps. Current methods require expert supervision and expensive runtime to evaluate the quality of intermediate outputs, limiting the usability and scalability in practical applications, such as wide area ground displacement monitoring and predication. We propose a deep convolutional neural network based model DeepInSAR to intelligently solve both phase filtering and coherence estimation problems. We demonstrate our model’s performance using simulated and real data. A teacher-student framework is introduced to handle the issue of missing clean InSAR ground truth. Quantitative and qualitative evaluations show that our teacher-student approach requires less input but can achieve better results than its stack-based teacher method even on new unseen data. The proposed DeepInSAR also outperforms three other top non-stack based methods in time efficiency without human supervision.

Phase filtering and pixel quality (coherence) estimation is critical in producing digital elevation models (DEMs) from interferometric synthetic aperture radar (InSAR) images, as it removes spatial inconsistencies (residues) and immensely improves the subsequent unwrapping. Large amount of InSAR data facilitates wide area monitoring (WAM) over geographical regions. Advances in parallel computing have accelerated convolutional neural networks (CNNs), giving them advantages over human performance on visual pattern recognition, which makes CNNs a good choice for WAM. Nevertheless, this research is largely unexplored. We thus propose “GenInSAR,” a CNN-based generative model for joint phase filtering and coherence estimation that directly learns the InSAR data distribution. GenInSAR’s unsupervised training on satellite and simulated noisy InSAR images outperforms other five related methods in total residue reduction (over 16(1/2)% better on average) with less over-smoothing/artifacts around branch cuts. GenInSAR’s phase and coherence root-mean-squared-error and phase cosine error have average improvements of 0.54, 0.07, and 0.05, respectively compared to the related methods.

Freezing of Gait (FOG) is an impairment that affects the majority of patients in the advanced stages of Parkinson’s Disease (PD). FOG can lead to sudden falls and injuries, negatively impacting the quality of life for the patients and their families. Rhythmic Auditory Stimulation (RAS) can be used to help patients recover from FOG and resume normal gait. RAS might be ineffective due to the latency between the start of a FOG event, its detection and initialization of RAS. We propose a system capable of both FOG prediction and detection using signals from tri-axial accelerometer sensors that will be useful in initializing RAS with minimal latency. We compared the performance of several time frequency analysis techniques, including moving windows extracted from the signals, handcrafted features, Recurrence Plots (RP), Short Time Fourier Transform (STFT), Discreet Wavelet Transform (DWT) and Pseudo Wigner Ville Distribution (PWVD) with Deep Learning (DL) based Long Short Term Memory (LSTM) and Convolutional Neural Networks (CNN). We also propose three Ensemble Network Architectures that combine all the time frequency representations and DL architectures. Experimental results show that our ensemble architectures significantly improve the performance compared with existing techniques. We also present the results of applying our method trained on a publicly available dataset to data collected from patients using wearable sensors in collaboration with A.T. Still University.