Xiao-Ping Zhang

Xiao-Ping Zhang

Auto-generated profile, based on publicly available information

Xiao-Ping Zhang

Tsinghua University

Research Focus & Metrics

Academic Career Length

26Years (2001-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
9 June 2020 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
Han XuHan Xu
Han Xu
Jiayi MaJiayi Ma
Jiayi Ma
XZXiao-Ping Zhang
Xiao-Ping Zhang
In this paper, we present an end-to-end architecture for multi-exposure image fusion based on generative adversarial networks, termed as MEF-GAN. In our architecture, a generator network and a discriminator network are trained simultaneously to form an adversarial relationship. The generator is trained to generate a real-like fused image based on the given source images which is expected to fool the discriminator. Correspondingly, the discriminator is trained to distinguish the generated fused images from the ground truth. The adversarial relationship makes the fused image not limited to the restriction of the content loss. Therefore, the fused images are closer to the ground truth in terms of probability distribution, which can compensate for the insufficiency of single content loss. Moreover, aiming at the problem that the luminance of multi-exposure images varies greatly with spatial location, the self-attention mechanism is employed in our architecture to allow for attention-driven and long-range dependency. Thus, local distortion, confusing results, or inappropriate representation can be corrected in the fused image. Qualitative and quantitative experiments are performed on publicly available datasets, where the results demonstrate that MEF-GAN outperforms the state-of-the-art, in terms of both visual effect and objective evaluation metrics. Our code is publicly available at https://github.com/jiayi-ma/MEF-GAN .

17 January 2020 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
XMXiongkuo Min
Xiongkuo Min
GZGuangtao Zhai
Guangtao Zhai
JZJiantao Zhou
Jiantao Zhou
XZXiao-Ping Zhang
Xiao-Ping Zhang
XYXiaokang Yang
Xiaokang Yang
XGXinping Guan
Xinping Guan
Audio information has been bypassed by most of current visual attention prediction studies. However, sound could have influence on visual attention and such influence has been widely investigated and proofed by many psychological studies. In this paper, we propose a novel multi-modal saliency (MMS) model for videos containing scenes with high audio-visual correspondence. In such scenes, humans tend to be attracted by the sound sources and it is also possible to localize the sound sources via cross-modal analysis. Specifically, we first detect the spatial and temporal saliency maps from the visual modality by using a novel free energy principle. Then we propose to detect the audio saliency map from both audio and visual modalities by localizing the moving-sounding objects using cross-modal kernel canonical correlation analysis, which is first of its kind in the literature. Finally we propose a new two-stage adaptive audiovisual saliency fusion method to integrate the spatial, temporal and audio saliency maps to our audio-visual saliency map. The proposed MMS model has captured the influence of audio, which is not considered in the latest deep learning based saliency models. To take advantages of both deep saliency modeling and audio-visual saliency modeling, we propose to combine deep saliency models and the MMS model via a later fusion, and we find that an average of 5% performance gain is obtained. Experimental results on audio-visual attention databases show that the introduced models incorporating audio cues have significant superiority over state-of-the-art image and video saliency models which utilize a single visual modality.

9 February 2024 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
YWYongzhen Wang
Yongzhen Wang
XYXuefeng Yan
Xuefeng Yan
Fu Lee WangFu Lee Wang
Fu Lee Wang
Haoran XieHaoran Xie
Haoran Xie
WYWenhan Yang
Wenhan Yang
Mingqiang WeiMingqiang Wei
Mingqiang Wei
While the wisdom of training an image dehazing model on synthetic hazy data can alleviate the difficulty of collecting real-world hazy/clean image pairs, it brings the well-known domain shift problem. From a different yet new perspective, this paper explores contrastive learning with an adversarial training effort to leverage unpaired real-world hazy and clean images, thus alleviating the domain shift problem and enhancing the network’s generalization ability in real-world scenarios. We propose an effective unsupervised contrastive learning paradigm for image dehazing, dubbed UCL-Dehaze. Unpaired real-world clean and hazy images are easily captured, and will serve as the important positive and negative samples respectively when training our UCL-Dehaze network. To train the network more effectively, we formulate a new self-contrastive perceptual loss function, which encourages the restored images to approach the positive samples and keep away from the negative samples in the embedding space. Besides the overall network architecture of UCL-Dehaze, adversarial training is utilized to align the distributions between the positive samples and the dehazed images. Compared with recent image dehazing works, UCL-Dehaze does not require paired data during training and utilizes unpaired positive/negative data to better enhance the dehazing performance. We conduct comprehensive experiments to evaluate our UCL-Dehaze and demonstrate its superiority over the state-of-the-arts, even only 1,800 unpaired real-world images are used to train our network. Source code is publicly available at https://github.com/yz-wang/UCL-Dehaze.

19 April 2023 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
TYTing Yao
Ting Yao
YLYehao Li
Yehao Li
YPYingwei Pan
Yingwei Pan
YWYu Wang
Yu Wang
XZXiao-Ping Zhang
Xiao-Ping Zhang
TMTao Mei
Tao Mei
Recent advances have presented several strategies to mitigate the computations of self-attention mechanism with high-resolution inputs. Many of these works consider decomposing the global self-attention procedure over image patches into regional and local feature extraction procedures that each incurs a smaller computational complexity. Despite good efficiency, these approaches seldom explore the holistic interactions among all patches, and are thus difficult to fully capture the global semantics. In this paper, we propose a novel Transformer architecture that elegantly exploits the global semantics for self-attention learning, namely Dual Vision Transformer (Dual-ViT). The new architecture incorporates a critical semantic pathway that can more efficiently compress token vectors into global semantics with reduced order of complexity. Such compressed global semantics then serve as useful prior information in learning finer local pixel level details, through another constructed pixel pathway. The semantic pathway and pixel pathway are integrated together and are jointly trained, spreading the enhanced self-attention information in parallel through both of the pathways. Dual-ViT is henceforth able to capitalize on global semantics to boost self-attention learning without compromising much computational complexity. We empirically demonstrate that Dual-ViT provides superior accuracy than SOTA Transformer architectures with comparable training complexity.

31 January 2022 in ACM Computing Surveys
Verified
Journal Article
Open access
HDHang Du
Hang Du
HSHailin Shi
Hailin Shi
DZDan Zeng
Dan Zeng
XZXiao-Ping Zhang
Xiao-Ping Zhang
TMTao Mei
Tao Mei
Face recognition (FR) is one of the most popular and long-standing topics in computer vision. With the recent development of deep learning techniques and large-scale datasets, deep face recognition has made remarkable progress and has been widely used in many real-world applications. Given a natural image or video frame as input, an end-to-end deep face recognition system outputs the face feature for recognition. To achieve this, a typical end-to-end system is built with three key elements: face detection, face alignment, and face representation. Face detection locates faces in the image or frame. Then, the face alignment is proceeded to calibrate the faces to the canonical view and crop them with a normalized pixel size. Finally, in the stage of face representation, the discriminative features are extracted from the aligned face for recognition. Nowadays, all of the three elements are fulfilled by the technique of deep convolutional neural network. In this survey article, we present a comprehensive review about the recent advance of each element of the end-to-end deep face recognition, since the thriving deep learning techniques have greatly improved their capability of them. To start with, we present an overview of the end-to-end deep face recognition. Then, we review the advance of each element, respectively, covering many aspects such as the to-date algorithm designs, evaluation metrics, datasets, performance comparison, existing challenges, and promising directions for future research. Also, we provide a detailed discussion about the effect of each element on its subsequent elements and the holistic system. Through this survey, we wish to bring contributions in two aspects: first, readers can conveniently identify the methods which are quite strong-baseline style in the subcategory for further exploration; second, one can also employ suitable methods for establishing a state-of-the-art end-to-end face recognition system from scratch.

1 May 2001 in IEEE Transactions on Neural Networks
Verified
Journal Article
Closed access
XZXiao-Ping Zhang
Xiao-Ping Zhang
In the paper, a type of thresholding neural network (TNN) is developed for adaptive noise reduction. New types of soft and hard thresholding functions are created to serve as the activation function of the TNN. Unlike the standard thresholding functions, the new thresholding functions are infinitely differentiable. By using the new thresholding functions, some gradient-based learning algorithms become possible or more effective. The optimal solution of the TNN in a mean square error (MSE) sense is discussed. It is proved that there is at most one optimal solution for the soft-thresholding TNN. General optimal performances of both soft and hard thresholding TNNs are analyzed and compared to the linear noise reduction method. Gradient-based adaptive learning algorithms are presented to seek the optimal solution for noise reduction. The algorithms include supervised and unsupervised batch learning as well as supervised and unsupervised stochastic learning. It is indicated that the TNN with the stochastic learning algorithms can be used as a novel nonlinear adaptive filter. It is proved that the stochastic learning algorithm is convergent in certain statistical sense in ideal conditions. Numerical results show that the TNN is very effective in finding the optimal solutions of thresholding methods in an MSE sense and usually outperforms other noise reduction methods. Especially, it is shown that the TNN-based nonlinear adaptive filtering outperforms the conventional linear adaptive filtering in both optimal solution and learning performance.