4 August 2025 in Nature Machine Intelligence
15 March 2024 in Pattern Recognition Letters
5 January 2024 in IEEE Sensors Journal
Deep neural network (DNN) has proven its powerful ability in feature representation and has been widely applied in computer vision tasks, especially optical flow estimation (OFE). However, these DNN-based OFE approaches have a black-box nature, resulting in lacking a clear interpretation for their success. More unfortunately, they usually suffer from the loss of objects’ shape, which is not beneficial for performance improvement. To address these issues, we develop a novel guided Taylor expansion approximation (GTEA) network, including the following key procedures: First, we present the detail analysis of the relationship between Taylor expansion and optical flow. Then, a convolutional guided filter (ConvGF) network is developed to refine the shape of objects. Finally, the final output is obtained by employing basic network, ConvGF, and detail network to approximate the Taylor expansion. Quantitatively and qualitatively experimental results on the Sintel and KITTI datasets demonstrate that the proposed GTEA is effective and even performs better than the state-of-the-art optical flow estimation methods on performance improvement and shape preservation.
28 September 2023 in IEEE Sensors Journal
Optical flow represents vector motion of targets, and its estimation is a hot research in computer vision. The traditional prior-based methods build mathematical models using different regularizers on object characteristics. These methods are straightforward and easily understood, but they suffer from a high computational burden. Deep learning based approaches greatly improve their efficiency, but their filter sizes are fixed and receptive fields are large. To solve these problems, this paper proposes a novel recurrent feature refinement network based on short-term dense connection modules for optical flow estimation (RFRFlow), including the following key parts: firstly, we utilize short-term dense concatenate modules (STDC) as a backbone part for constructing a feature extracting network which can preserve rich details and use global averaging pooling to extract global context information with a large receptive field. Secondly, a 4D correlation volume is constructed using context features of image pairs to extract multi-scale 4D correlation information. Thirdly, a motion feature encoder is reported to transfer the multi-scale 4D correlation information into 2D motion features. Finally, 2D context features and 2D motion features are input into a global motion aggregation (GMA) decoder for pursuing the final optical flow. In the experiments, we firstly present and analyze the results of ablation studies on the selection of batch size and image size. Then, both quantitative and qualitative results on Sintel and KITTI datasets are compared, showing that the proposed RFRFlow is effective and even performs better than the state-of-the-art optical flow estimation methods on performance improvement and objects’ shape preservation.
14 April 2022 in IEEE Geoscience and Remote Sensing Letters
Remotely sensed images degraded by additive white Gaussian noise (AWGN) are not beneficial for the analysis of their contents. Such a phenomenon is usually modeled as an inverse problem which can be solved by model-based optimization methods or discriminative learning approaches. The former pursue their pleasing performance at the cost of a highly computational burden while the latter are impressive for their fast testing speed but are limited by their application range. To join their merits, this letter proposes a nonlocal self-similar (NSS) block-based deep image denoising scheme, namely deep low-rank prior (DLRP), which includes the following key points: First, the low-rank property of the neighboring NSS patches ordered lexicographically is utilized to model a global objective function (GOF). Second, with the aid of an alternative iteration strategy, the GOF can be easily decomposed into two independent subproblems. One is a quadratic optimization problem, and has a closed-form solution. While the other is a low-rank minimization denoising problem and is learned by deep convolutional neural network (DCNN). Then, the deep denoiser, acted as a modular part, is plugged into the model-based optimization method with adaptive noise level estimation to solve the inverse problem. In the experiments, we first discuss parameter setting and the convergence. Then, quantitative/qualitative comparisons of experimental results validate that the DLRP is a flexible and powerful denoising method to achieve competitive performance which even outperforms those produced by state-of-the-arts.