18 November 2020 in IEEE Access
In recent years, the excellent image-based object detection algorithms are transferred to the video object detection directly. These frame-by-frame processing methods are suboptimal owing to the degenerate object appearance such as motion blur, defocus and rare poses. The existing works for video object detection mostly focus on the feature aggregation at pixel level and instance level, but the blur impact in the aggregation process has not been exploited well so far. In this paper, we propose an end-to-end blur-aid feature aggregation network (BFAN) for video object detection. The proposed BFAN focuses on the aggregation process influenced by the blur including motion blur and defocus with high accuracy and little increased computation. In BFAN, we evaluate the object blur degree of each frame as the weight for aggregation. Noteworthy, the background is usually flat which has a negative impact on the object blur degree evaluation. Therefore, we introduce a light saliency detection network to alleviate the background interference. The experiments conducted on the ImageNet VID dataset show that BFAN achieves the state-of-the-art detection performance, exactly 79.1% mAP, with 3 points improvement compared to the video object detection baseline.
1 July 2020 in Neurocomputing
14 July 2021 in IEEE Signal Processing Letters
Motion blur is an inevitable problem when shooting a video in motion, and it will cause serious degradation of images. In this letter, we propose a new robust incremental weighted Fourier aggregation algorithm for dynamic scene video deblurring. This is motivated by the fact that existing multiframe aggregation methods usually require a tedious iterative process to obtain sharp information in distant frames. We propose to perform alignment and aggregation once for each frame using the current frame and the previous frame result, which includes the sharp information of all the previous frames. Moreover, a criterion for evaluating the alignment of blurred images is proposed. Based on this, a robust aggregation approach is also proposed to help reduce artifacts caused by image patch alignment failure. Specifically, we decompose the current frame into a series of image patches, find consistent image patches in the deblurring results of the previous frame by a coarse-to-fine alignment algorithm, and aggregate them in the Fourier domain. Experiments show that our method can obtain improved deblurring results and reduce the computational complexity compared to the traditional aggregation method.