Although recurrent network-based optical flow estimation methods have shown great success in recent years, most of these methods have difficulty handling large displacements and occlusions because the existing recurrent networks are usually restricted to coarse-resolution single-scale models while ignoring the multiscale features brought by hierarchical concepts in previous coarse-to-fine approaches. In this paper, we propose an adaptive-aware correlation recurrent network for optical flow estimation, named ACR-Net, which preserves fine motion features with a single-scale resolution recurrent framework and adaptively incorporates multiscale features at different stages to achieve high-accuracy optical flow estimation. First, our proposed self-adaptation scale-aware correlation module can incorporate the adaptive correlation of multiscale inter- and intra-motion features, which makes the features more discriminative for capturing long-range dependencies between pixels. Second, our presented adaptive-aware motion module can effectively extract the required features of different kinds of motion from multilevel correspondence. Third, our introduced cross-guide motion and fusion modules can accurately guide the propagation of reliable pixels towards unreliable pixels and dynamically determine the most suitable expression to address the occlusion challenges. Comprehensive experiments demonstrate that ACR-Net outperforms existing two-view models, striking a good balance between speed and accuracy and achieving the best performance on the MPI-Sintel final pass and KITTI-2015 test datasets. The code will be made publicly available.
28 February 2024 in IEEE Transactions on Instrumentation and Measurement
Although deep learning-based surface defect detection approaches have performed remarkably well in recent years, the complicated shapes and large size differences of surface defects still pose enormous challenges for most existing methods. To address these issues, we propose a novel surface defect detection method joining spatial deformable convolution and a dense feature pyramid, named SDDF-Net. First, we construct a spatial deformable convolution-based feature extraction network, which uses a dynamic convolutional kernel with spatial information to increase the feature extraction capability of complicated defects. Second, we build a dense feature pyramid-based feature fusion network that fuses features from different network layers to improve the detection accuracy of multiscale defects. Third, we present a novel hybrid loss that combines complete intersection over union loss and normalized Wasserstein distance loss to enhance the defect recognition and location learning abilities of our method. Finally, we run our method on the NEU-DET, DAGM2007 and DeepPCB datasets to conduct a comprehensive comparison with some state-of-the-art general object detection models and specialized surface defect detection methods. The experimental results show that the proposed SDDF-Net performs competitively in terms of detection accuracy and computational efficiency when compared with existing methods. This indicates that SDDF-Net achieves good results for surface defect detection and is qualified for real-time processing in industry.
17 July 2023 in IEEE Transactions on Geoscience and Remote Sensing
Infrared small target detection plays an important role in many military and civilian applications. Despite the great advances made by infrared small target detection studies in recent years, most of the existing methods have difficulty in balancing detection probabilities and false alarms. Moreover, there are only a few public datasets for infrared small targets, which limits the development of infrared small target detection research. To address the abovementioned issues, in this paper, we propose a robust infrared small target detection method that joins multiple pieces of information and noise predictions, named MINP-Net. Specifically, we first design a gradient and contextual information extraction module to extract multiscale features from an input infrared image. Second, we construct a noise prediction network to model the background noise. Third, we plan a regional positioning branch to provide a coarse target location to decrease the false alarm ratio. In addition, we build a new infrared small target detection benchmark to advance the research in this field, named the NCHU-Seg dataset. To the best of our knowledge, the NCHU-Seg dataset is the largest real-world scene dataset for evaluating infrared small target segmentation methods. For a comprehensive evaluation, we compare our method with some of the state-of-the-art methods on both the well-known NUAA-SIRST dataset and our NCHU-Seg dataset. The experimental results demonstrate that the proposed MINP-Net method performs better in terms of detection effectiveness and segmentation accuracy and effectively balances the detection probabilities and false alarms with complex backgrounds. (The code and dataset are available at https://github.com/PCwenyue.).
5 June 2017 in IEEE Transactions on Image Processing
In this paper, we propose a robust non-local TV- $L^{1}$ optical flow method with occlusion detection to address the problem of weak robustness of optical flow estimation with motion occlusion. First, a TV- $L^{1}$ form for flow estimation is defined using a combination of the brightness constancy and gradient constancy assumptions in the data term and by varying the weight under the Charbonnier function in the smoothing term. Second, to handle the potential risk of the outlier in the flow field, a general non-local term is added in the TV- $L^{1}$ optical flow model to engender the typical non-local TV- $L^{1}$ form. Third, an occlusion detection method based on triangulation is presented to detect the occlusion regions of the sequence. The proposed non-local TV- $L^{1}$ optical flow model is performed in a linearizing iterative scheme using improved median filtering and a coarse-to-fine computing strategy. The results of the complex experiment indicate that the proposed method can overcome the significant influence of non-rigid motion, motion occlusion, and large displacement motion. Results of experiments comparing the proposed method and existing state-of-the-art methods by, respectively, using Middlebury and MPI Sintel database test sequences show that the proposed method has higher accuracy and better robustness.
There exist many multi-objective optimization problems (MOPs) containing several inequality and equality constraints in practical applications, which are known as CMOPs. CMOPs pose great challenges for existing multi-objective evolutionary algorithms (MOEAs) since the difficulty in balancing the objective minimization and constraint satisfaction. Without loss of generality, the distribution of the Pareto set for a continuous m-objective CMOP can be regarded as a piecewise continuous manifold of dimension (m − 1). According to this property, a self-organizing map (SOM) approach for constrained multi-objective optimization problems is proposed in this article. In the proposed approach, we adopt the strategy of two population evolution, in which one population is evolved by considering all the constraints and the other population is used to assist in exploring the areas. In the evolutionary stage, each population is assigned a self-organizing map for discovering the population distribution structure in the decision space. After the topological mapping, we utilize the extracted neighborhood relationship information to generate promising offspring solutions. Afterwards, the neuron weight vectors of SOM are updated by the objective vectors of the surviving offsprings. Through the proposed approach, we can make the population efficiently converge to the feasible region with suitable levels of diversity. In the experiments, we compare the proposed method with several state-of-the-art approaches by using 48 benchmark problems. The evaluation results indicate that the overwhelmingly superior performance of the proposed method over the other peer algorithms on most of the tested problems. The source code is available at https://github.com/hccccc92918/CMOSMA.