Deep unfolding methods design deep neural networks as learned variations of optimization algorithms through the unrolling of their iterations. These networks have been shown to achieve faster convergence and higher accuracy than the original optimization methods. In this line of research, this paper presents novel interpretable deep recurrent neural networks (RNNs), designed by the unfolding of iterative algorithms that solve the task of sequential signal reconstruction (in particular, video reconstruction). The proposed networks are designed by accounting that video frames’ patches have a sparse representation and the temporal difference between consecutive representations is also sparse. Specifically, we design an interpretable deep RNN (coined reweighted-RNN) by unrolling the iterations of a proximal method that solves a reweighted version of the $\ell _{1}$ - $\ell _{1}$ minimization problem. Due to the underlying minimization model, our reweighted-RNN has a different thresholding function (alias, different activation function) for each hidden unit in each layer. In this way, it has higher network expressivity than existing deep unfolding RNN models. We also present the derivative $\ell _{1}$ - $\ell _{1}$ -RNN model, which is obtained by unfolding a proximal method for the $\ell _{1}$ - $\ell _{1}$ minimization problem. We apply the proposed interpretable RNNs to the task of video frame reconstruction from low-dimensional measurements, that is, sequential video frame reconstruction. The experimental results on various datasets demonstrate that the proposed deep RNNs outperform various RNN models.

Transform Domain Wyner-Ziv (TDWZ) video coding is an efficient Distributed Video coding solution providing new features such as low complexity encoding, by mainly exploiting the source statistics at the decoder based on the availability of decoder side information. The accuracy of the decoder side information has a major impact on the performance of TDWZ. In this paper, a novel multi-hypothesis based TDWZ video coding is presented to exploit the redundancy between multiple side information and the source information. The decoder used optical flow for side information calculation. Compared with the best available single estimation mode TDWZ, the proposed multi-hypothesis based TDWZ achieves robustly better Rate-Distortion (RD) performance and the overall improvement is up to 0.6 dB at high bitrate and up to 2 dB compared with the DISCOVER TDWZ video codec.

In the context of video background–foreground separation, we propose a compressive online Robust Principal Component Analysis (RPCA) with optical flow that separates recursively a sequence of video frames into foreground (sparse) and background (low-rank) components. This separation method operates on a small set of measurements taken per frame, in contrast to conventional batch-based RPCA, which processes the full data. The proposed method also leverages multiple prior information by incorporating previously separated background and foreground frames in an n-${\ell }_1$ minimization problem. Moreover, optical flow is utilized to estimate motions between the previous foreground frames and then compensate the motions to achieve higher quality prior foregrounds for improving the separation. Our method is tested on several video sequences in different scenarios for online background–foreground separation given compressive measurements. The visual and quantitative results show that the proposed method outperforms other existing methods.

This work considers online robust principal component analysis (RPCA) in time-varying decomposition problems such as video foreground-background separation. We propose a compressive online RPCA algorithm that decomposes recursively a sequence of data vectors (e.g., frames) into sparse and lowrank components. Different from conventional batch RPCA, which processes all the data directly, our approach considers a small set of measurements taken per data vector (frame). Moreover, our algorithm can incorporate multiple prior information from previous decomposed vectors via proposing an n-ℓ1 minimization method. At each time instance, the algorithm recovers the sparse vector by solving the n-ℓ1 minimization problem—which promotes not only the sparsity of the vector but also its correlation with multiple previously-recovered sparse vectors—and, subsequently, updates the low-rank component using incremental singular value decomposition.We also establish theoretical bounds on the number of measurements required to guarantee successful compressive separation under the assumptions of static or slowly-changing low-rank components. We evaluate the proposed algorithm using numerical experiments and online video foreground-background separation experiments. The experimental results show that the proposed method outperforms the existing methods.

This paper studies an advanced machine learning method, specifically few-shot classification with meta-learning, applied to distributed acoustic sensing (DAS) data. The study contributes two key aspects: (i) an investigation of different pre-processing methods for DAS data and (ii) the implementation of a neural network model based on meta-learning to learn a representation of the processed data. In the context of urban infrastructure monitoring, we develop a few-shot classification framework that classifies query samples with only a limited number of support samples. The model consists of an embedding network trained on a meta dataset for feature extraction and is followed by a classifier for performing few-shot classification. This research thoroughly explores three types of data pre-processing, that is, decomposed phase, power spectral density, and frequency energy band, as inputs to the neural network. Experimental results show the efficient learning capabilities of the embedding model when working with various pre-processed data, offering a range of pre-processing options. Furthermore, the results demonstrate outstanding few-shot classification performance across a large number of event classes, highlighting the framework’s potential for urban infrastructure monitoring applications.

We propose a new deep recurrent neural network (RNN) architecture for sequential signal reconstruction. Our network is designed by unfolding the iterations of the proximal gradient method that solves the ℓ 1 -ℓ 1 minimization problem. As such, our network leverages by design that signals have a sparse representation and that the difference between consecutive signal representations is also sparse. We evaluate the proposed model in the task of reconstructing video frames from compressive measurements and show that it outperforms several state-of-the-art RNN models.