This page has only limited features, please log in for full access.

Unclaimed
Fan Li
School of Electronic and Information Engneering, Xi'an Jiaotong University, 12480 Xi'an, Shaanxi, China

Honors and Awards

The user has no records in this section


Career Timeline

The user has no records in this section.


Short Biography

The user biography is not available.
Following
Followers
Co Authors
The list of users this user is following is empty.
Following: 0 users

Feed

Journal article
Published: 30 August 2021 in IEEE Transactions on Instrumentation and Measurement
Reads 0
Downloads 0

How best to measure spatial saliency shift induced by image distortions is an open research question. Our previous study has shown that image distortions cause saliency to deviate from its original places in natural images, and the degree of such distortion-induced saliency variation (DSV) depends on image content as well as the properties of distortion. Being able to measure DSV benefits the development of saliency based image quality algorithms. In this paper, we first investigate the plausibility of using existing mathematical algorithms for measuring DSV and their potential limitations. We then develop a new algorithm for quantifying DSV, based on a deep neural network. In the algorithm, namely ST-DSV, we design a coarse-grained to fine-grained saliency similarity transformation approach to achieve DSV measurement. The experimental results show that the proposed ST-DSV algorithm significantly outperforms existing methods in predicting the ground truth DSV.

ACS Style

Xiaohan Yang; Fan Li; Hantao Liu. A Measurement for Distortion Induced Saliency Variation in Natural Images. IEEE Transactions on Instrumentation and Measurement 2021, PP, 1 -1.

AMA Style

Xiaohan Yang, Fan Li, Hantao Liu. A Measurement for Distortion Induced Saliency Variation in Natural Images. IEEE Transactions on Instrumentation and Measurement. 2021; PP (99):1-1.

Chicago/Turabian Style

Xiaohan Yang; Fan Li; Hantao Liu. 2021. "A Measurement for Distortion Induced Saliency Variation in Natural Images." IEEE Transactions on Instrumentation and Measurement PP, no. 99: 1-1.

Journal article
Published: 30 August 2021 in IEEE Transactions on Vehicular Technology
Reads 0
Downloads 0

Caching and transcoding at multi-access edge computing (MEC) server and wireless resource allocation in eNodeB interact with each other and together determine the quality of experience (QoE) of dynamic adaptive streaming over HTTP (DASH) clients. However, the relationship among the three factors has not been explored, which has led to limited improvement in clients' QoE. Therefore, we propose a joint optimization framework of video segment caching and transcoding in MEC servers and resource allocation to improve the QoE of DASH clients. Based on the established framework, we develop an MEC caching management mechanism that consists of the MEC caching partition, video segment deletion, and MEC caching space transfer. Then, a joint optimization algorithm that combines the video segment caching and transcoding in the MEC server and resource allocation is proposed. In the algorithm, the clients' channel state and the playback status and cooperation among MEC servers are employed to estimate the client's priority, video segment representation switch and continuous playback time. Considering the above four factors, we develop a utility function model of clients' QoE. Then, we formulate a mixed-integer nonlinear programming mathematical model to maximize the total utility of DASH clients, where the video segment caching and transcoding strategy and resource allocation strategy are jointly optimized. To solve this problem, we propose a low-complexity heuristic algorithm that decomposes the original problem into multiple subproblems. The simulation results show that our proposed algorithms efficiently improve client's throughput, received video quality and hit ratio of video segments while decreasing the playback rebuffering time, video segment representation switch and system backhaul traffic.

ACS Style

Xinyu Huang; Lijun He; Liejun Wang; Fan Li. Towards 5G: Joint Optimization of Video Segment Caching, Transcoding and Resource Allocation for Adaptive Video Streaming in a Multi-access Edge Computing Network. IEEE Transactions on Vehicular Technology 2021, PP, 1 -1.

AMA Style

Xinyu Huang, Lijun He, Liejun Wang, Fan Li. Towards 5G: Joint Optimization of Video Segment Caching, Transcoding and Resource Allocation for Adaptive Video Streaming in a Multi-access Edge Computing Network. IEEE Transactions on Vehicular Technology. 2021; PP (99):1-1.

Chicago/Turabian Style

Xinyu Huang; Lijun He; Liejun Wang; Fan Li. 2021. "Towards 5G: Joint Optimization of Video Segment Caching, Transcoding and Resource Allocation for Adaptive Video Streaming in a Multi-access Edge Computing Network." IEEE Transactions on Vehicular Technology PP, no. 99: 1-1.

Journal article
Published: 26 August 2021 in Remote Sensing
Reads 0
Downloads 0

Although considerable success has been achieved in change detection on optical remote sensing images, accurate detection of specific changes is still challenging. Due to the diversity and complexity of the ground surface changes and the increasing demand for detecting changes that require high-level semantics, we have to resort to deep learning techniques to extract the intrinsic representations of changed areas. However, one key problem for developing deep learning metho for detecting specific change areas is the limitation of annotated data. In this paper, we collect a change detection dataset with 862 labeled image pairs, where the urban construction-related changes are labeled. Further, we propose a supervised change detection method based on a deep siamese semantic segmentation network to handle the proposed data effectively. The novelty of the method is that the proposed siamese network treats the change detection problem as a binary semantic segmentation task and learns to extract features from the image pairs directly. The siamese architecture as well as the elaborately designed semantic segmentation networks significantly improve the performance on change detection tasks. Experimental results demonstrate the promising performance of the proposed network compared to existing approaches.

ACS Style

Le Yang; Yiming Chen; Shiji Song; Fan Li; Gao Huang. Deep Siamese Networks Based Change Detection with Remote Sensing Images. Remote Sensing 2021, 13, 3394 .

AMA Style

Le Yang, Yiming Chen, Shiji Song, Fan Li, Gao Huang. Deep Siamese Networks Based Change Detection with Remote Sensing Images. Remote Sensing. 2021; 13 (17):3394.

Chicago/Turabian Style

Le Yang; Yiming Chen; Shiji Song; Fan Li; Gao Huang. 2021. "Deep Siamese Networks Based Change Detection with Remote Sensing Images." Remote Sensing 13, no. 17: 3394.

Journal article
Published: 21 July 2021 in Journal of Visual Communication and Image Representation
Reads 0
Downloads 0

While deep learning-based image compression methods have shown impressive coding performance, most existing methods are still in the mire of two limitations: (1) unpredictable compression efficiency gain when adopting convolutional neural networks with different depths, and (2) lack of an accurate model to estimate the entropy during the training process. To address these two problems, in this paper, a deep multi-stage representation based image compression (MSRIC) method is proposed. Owing to this architecture, the detail information of shallow stages and the compact information of deep stages can be utilized for image reconstruction. Furthermore, a data-dependent channel-wised factorized probability model (DCFPM) is adopted to increase the accuracy of entropy estimation. Experimental results indicate that the proposed method guarantees better perceptual performance at a wide range of bit-rates. Also, ablation studies are carried out to validate the above mentioned technologies.

ACS Style

Zixi Wang; Guiguang Ding; Jungong Han; Fan Li. Deep image compression with multi-stage representation. Journal of Visual Communication and Image Representation 2021, 79, 103226 .

AMA Style

Zixi Wang, Guiguang Ding, Jungong Han, Fan Li. Deep image compression with multi-stage representation. Journal of Visual Communication and Image Representation. 2021; 79 ():103226.

Chicago/Turabian Style

Zixi Wang; Guiguang Ding; Jungong Han; Fan Li. 2021. "Deep image compression with multi-stage representation." Journal of Visual Communication and Image Representation 79, no. : 103226.

Article
Published: 03 June 2021 in Applied Intelligence
Reads 0
Downloads 0

Surface defect inspection can greatly improve the efficiency of industrial production by replacing manual operations. However, in actual industrial scenarios, it is difficult to collect and manually label enough defect images. In addition, the complex backgrounds, diverse shapes and sizes, and broad random location distribution of defects in images make defect inspection more challenging. To address these issues, we propose an unsupervised defect inspection algorithm based on cascaded GAN (Generative Adversarial Networks) with edge repair feature fusion. In this algorithm, the edge repair network provides intact structural features for the defect repair network by means of a feature fusion method based on channel attention. For the edge repair network, we develop a deformable autoencoder, which fully utilizes the ability of deformable convolution to perceive very little contextual information to improve its ability to repair defect edges. Specifically, training requires only a few defect-free images and no labeled defect images. To verify the effectiveness of the proposed algorithm, we compare it with existing algorithms in terms of precision, the F1-measure, and the mIoU (mean Intersection over Union). The experimental results show that the proposed algorithm exhibits better defect inspection performance, especially for defects with rich forms and diverse positions against complex backgrounds.

ACS Style

Lijun He; Nan Shi; Kainnat Malik; Fan Li. Unsupervised defect inspection algorithm based on cascaded GAN with edge repair feature fusion. Applied Intelligence 2021, 1 -19.

AMA Style

Lijun He, Nan Shi, Kainnat Malik, Fan Li. Unsupervised defect inspection algorithm based on cascaded GAN with edge repair feature fusion. Applied Intelligence. 2021; ():1-19.

Chicago/Turabian Style

Lijun He; Nan Shi; Kainnat Malik; Fan Li. 2021. "Unsupervised defect inspection algorithm based on cascaded GAN with edge repair feature fusion." Applied Intelligence , no. : 1-19.

Journal article
Published: 28 January 2021 in IEEE Transactions on Circuits and Systems for Video Technology
Reads 0
Downloads 0

As the evaluation of image quality depends on the human visual system (HVS), many existing image quality assessment (IQA) methods focus on modeling the HVS to account for subjective perception. The visual attention of the HVS makes humans more sensitive to distortion on the attended regions than on regions which are not the focus of attention. Therefore, we propose an end-to-end multi-task deep convolution neural network with multi-scale and multi-hierarchy fusion (MMMNet), in which the IQA and saliency subtasks are jointly optimized to improve saliency-guided IQA performance. Particularly, the incorporation of saliency information is achieved by fusing saliency features with IQA features hierarchically to progressively improve the IQA features over network depth. A multi-scale feature extraction module (MSFE) is proposed to provide effective saliency features for the IQA network. Based on the saliency fusion, MMMNet introduces an auxiliary saliency task, achieving the multi-task learning to improve the generalization of the IQA task. Experimental results show that MMMNet achieves state-of-the-art performance and strong generalization ability on IQA databases.

ACS Style

Fan Li; Yangfan Zhang; Pamela C. Cosman. MMMNet: An End-to-End Multi-task Deep Convolution Neural Network with Multi-scale and Multi-hierarchy Fusion for Blind Image Quality Assessment. IEEE Transactions on Circuits and Systems for Video Technology 2021, PP, 1 -1.

AMA Style

Fan Li, Yangfan Zhang, Pamela C. Cosman. MMMNet: An End-to-End Multi-task Deep Convolution Neural Network with Multi-scale and Multi-hierarchy Fusion for Blind Image Quality Assessment. IEEE Transactions on Circuits and Systems for Video Technology. 2021; PP (99):1-1.

Chicago/Turabian Style

Fan Li; Yangfan Zhang; Pamela C. Cosman. 2021. "MMMNet: An End-to-End Multi-task Deep Convolution Neural Network with Multi-scale and Multi-hierarchy Fusion for Blind Image Quality Assessment." IEEE Transactions on Circuits and Systems for Video Technology PP, no. 99: 1-1.

Journal article
Published: 13 January 2021 in Neurocomputing
Reads 0
Downloads 0

Smoke detection based on video monitoring is of great importance for early fire warning. However, most of the smoke detection methods based on neural network only consider the normal weather. The harsh weather such as the fog environment is ignored. In this paper, we propose a smoke detection in normal and fog weather, which combines attention mechanism and feature-level and decision-level fusion module. First, a new fog smoke dataset with diverse positive and hard negative samples dataset is established through online collection and offline shooting. Then, an attention mechanism module combining spatial attention and channel attention is proposed to solve the problem of small smoke detection. Next, a lightweight feature-level and decision-level fusion module is proposed, which can not only improve the discrimination of smoke, fog and other similar objects, but also ensure the real-time performance of the model. Finally, a large number of comparative experiments on the existing dataset and our self-created dataset, show that our method can obtain higher detection accuracy rate, precision rate, recall rate, and F1 score from the perspective of overall, each category, small smoke and hard negative samples detection than the existing methods.

ACS Style

Lijun He; Xiaoli Gong; Sirou Zhang; Liejun Wang; Fan Li. Efficient attention based deep fusion CNN for smoke detection in fog environment. Neurocomputing 2021, 434, 224 -238.

AMA Style

Lijun He, Xiaoli Gong, Sirou Zhang, Liejun Wang, Fan Li. Efficient attention based deep fusion CNN for smoke detection in fog environment. Neurocomputing. 2021; 434 ():224-238.

Chicago/Turabian Style

Lijun He; Xiaoli Gong; Sirou Zhang; Liejun Wang; Fan Li. 2021. "Efficient attention based deep fusion CNN for smoke detection in fog environment." Neurocomputing 434, no. : 224-238.

Journal article
Published: 14 December 2020 in IEEE Transactions on Image Processing
Reads 0
Downloads 0

Intra/inter switching-based error resilient video coding effectively enhances the robustness of video streaming when transmitting over error-prone networks. But it has a high computation complexity, due to the detailed end-to-end distortion prediction and brute-force search for rate-distortion optimization. In this paper, a Low Complexity Mode Switching based Error Resilient Encoding (LC-MSERE) method is proposed to reduce the complexity of the encoder through a deep learning approach. By designing and training multi-scale information fusion-based convolutional neural networks (CNN), intra and inter mode coding unit (CU) partitions can be predicted by the networks rapidly and accurately, instead of using brute-force search and a large number of end-to-end distortion estimations. In the intra CU partition prediction, we propose a spatial multi-scale information fusion based CNN (SMIF-Intra). In this network a shortcut convolution architecture is designed to learn the multi-scale and multi-grained image information, which is correlated with the CU partition. In the inter CU partition, we propose a spatial-temporal multi-scale information fusion-based CNN (STMIF-Inter), in which a two-stream convolution architecture is designed to learn the spatial-temporal image texture and the distortion propagation among frames. With information from the image, and coding and transmission parameters, the networks are able to accurately predict CU partitions for both intra and inter coding tree units (CTUs). Experiments show that our approach significantly reduces computation time for error resilient video encoding with acceptable quality decrement.

ACS Style

Taiyu Wang; Fan Li; Xiaoya Qiao; Pamela C. Cosman. Low-Complexity Error Resilient HEVC Video Coding: A Deep Learning Approach. IEEE Transactions on Image Processing 2020, 30, 1245 -1260.

AMA Style

Taiyu Wang, Fan Li, Xiaoya Qiao, Pamela C. Cosman. Low-Complexity Error Resilient HEVC Video Coding: A Deep Learning Approach. IEEE Transactions on Image Processing. 2020; 30 (99):1245-1260.

Chicago/Turabian Style

Taiyu Wang; Fan Li; Xiaoya Qiao; Pamela C. Cosman. 2020. "Low-Complexity Error Resilient HEVC Video Coding: A Deep Learning Approach." IEEE Transactions on Image Processing 30, no. 99: 1245-1260.

Original research paper
Published: 06 December 2020 in IET Image Processing
Reads 0
Downloads 0
ACS Style

Hong Zhang; Fan Li; Zhisheng Yan. Feature fusion quality assessment model for DASH video streaming. IET Image Processing 2020, 15, 127 -142.

AMA Style

Hong Zhang, Fan Li, Zhisheng Yan. Feature fusion quality assessment model for DASH video streaming. IET Image Processing. 2020; 15 (1):127-142.

Chicago/Turabian Style

Hong Zhang; Fan Li; Zhisheng Yan. 2020. "Feature fusion quality assessment model for DASH video streaming." IET Image Processing 15, no. 1: 127-142.

Journal article
Published: 25 November 2020 in IEEE Transactions on Multimedia
Reads 0
Downloads 0

Image quality assessment (IQA) based on deep learning faces the overfitting problem due to limited training samples available in existing IQA databases. Transfer learning is a plausible solution to the problem, in which the shared features derived from the large-scale Imagenet source domain could be transferred from the original recognition task to the intended IQA task. However, the Imagenet source domain and the IQA target domain as well as their corresponding tasks are not directly related. In this paper, we propose a new transitive transfer learning method for no-reference image quality assessment (TTL-IQA). First, the architecture of the multi-domain transitive transfer learning for IQA is developed to transfer the Imagenet source domain to the auxiliary domain, and then to the IQA target domain. Second, the auxiliary domain and the auxiliary task are constructed by a new generative adversarial network based on distortion translation (DT-GAN). Furthermore, a TTL network of the semantic features transfer (SFTnet) is proposed to optimize the shared features for the TTL-IQA. Experiments are conducted to evaluate the performance of the proposed method on various IQA databases, including the LIVE, TID2013, CSIQ, LIVE multiply distorted and LIVE challenge. The results show that the proposed method significantly outperforms the state-of-the-art methods. In addition, our proposed method demonstrates a strong generalization ability.

ACS Style

Xiaohan Yang; Fan Li; Hantao Liu. TTL-IQA: Transitive Transfer Learning based no-reference Image Quality Assessment. IEEE Transactions on Multimedia 2020, PP, 1 -1.

AMA Style

Xiaohan Yang, Fan Li, Hantao Liu. TTL-IQA: Transitive Transfer Learning based no-reference Image Quality Assessment. IEEE Transactions on Multimedia. 2020; PP (99):1-1.

Chicago/Turabian Style

Xiaohan Yang; Fan Li; Hantao Liu. 2020. "TTL-IQA: Transitive Transfer Learning based no-reference Image Quality Assessment." IEEE Transactions on Multimedia PP, no. 99: 1-1.

Article
Published: 28 October 2020 in Applied Intelligence
Reads 0
Downloads 0

Frequent vehicle thefts have a highly detrimental impact on public safety. Thanks to surveillance equipment distributed throughout a city, a large number of videos that can be used to recognize vehicle theft are available. However, vehicle theft behavior has the characteristics of a small criminal target and small movement. Hence, the existing action recognition algorithms cannot be directly applied for the recognition of vehicle theft. In this paper, we propose a method for vehicle theft recognition based on a spatiotemporal attention mechanism. First, a database of vehicle theft is established by collecting videos from the Internet and an existing dataset. Then, we establish a vehicle theft recognition network and introduce a spatiotemporal attention mechanism for application when extracting the spatiotemporal features of theft. Through the learning of adaptive feature weights, the features that contribute most greatly to recognition are emphasized. Simulation experiments show that our proposed algorithm can achieve 97.04% accuracy on the collected vehicle theft database.

ACS Style

Lijun He; Shuai Wen; Liejun Wang; Fan Li. Vehicle theft recognition from surveillance video based on spatiotemporal attention. Applied Intelligence 2020, 51, 2128 -2143.

AMA Style

Lijun He, Shuai Wen, Liejun Wang, Fan Li. Vehicle theft recognition from surveillance video based on spatiotemporal attention. Applied Intelligence. 2020; 51 (4):2128-2143.

Chicago/Turabian Style

Lijun He; Shuai Wen; Liejun Wang; Fan Li. 2020. "Vehicle theft recognition from surveillance video based on spatiotemporal attention." Applied Intelligence 51, no. 4: 2128-2143.

Article
Published: 15 September 2020 in Multimedia Tools and Applications
Reads 0
Downloads 0

Video coding is one of the key technologies of visual sensors. As the state-of-art video coding standard, High Efficiency Video Coding (HEVC) achieves a significant high compression ratio for video. However, it also introduces heavy computational complexity, leading to challenges in application of visual sensors. To reduce the complexity of HEVC intra encoder, this paper proposed a one-stage decision method of CU/PU partition and prediction mode for intra coding. First, the potential factors that may related to the corresponding decisions in CU/PU are explored. Based on this, a one-stage decision network (OSDN) structure is specially designed to determine these decisions. Consequently, the complexity of HEVC intra coding can be drastically reduced by avoiding the brute-force search. Then, OSDN is embedded into the HEVC reference software HM 15.0. Thresholds are set to let the encoder switch between OSDN and the original implementation in HEVC to obtain the final decisions. The experimental results show that the proposed method can reduce 73.69% intra encoding time with 0.1673 dB BD-PSNR loss on average. In addition, the trade-off between RD performance degradation and complexity reduction can be controlled by thresholds.

ACS Style

Zixi Wang; Fan Li. Convolutional neural network based low complexity HEVC intra encoder. Multimedia Tools and Applications 2020, 80, 2441 -2460.

AMA Style

Zixi Wang, Fan Li. Convolutional neural network based low complexity HEVC intra encoder. Multimedia Tools and Applications. 2020; 80 (2):2441-2460.

Chicago/Turabian Style

Zixi Wang; Fan Li. 2020. "Convolutional neural network based low complexity HEVC intra encoder." Multimedia Tools and Applications 80, no. 2: 2441-2460.

Article
Published: 23 July 2020 in Multimedia Tools and Applications
Reads 0
Downloads 0

Deep learning-based methods have recently attracted significant attention in visual tracking community, leading to an increase in state-of-the-art tracking performance. However, due to the utilization of more complex models, it has also been accompanied with a decrease in speed. For real-time tracking applications, a careful balance of performance and speed is required. We propose a real-time tracking method based on deep feature fusion, which combines deep learning with kernel correlation filter. First, hierarchical features are extracted from a lightweight pre-trained convolutional neural network. Then, original features of different levels are fused using canonical correlation analysis. Fused features, as well as some original deep features, are used in three kernel correlation filters to track the target. An adaptive update strategy, based on dispersion analysis of response maps for the correlation filters, is proposed to improve robustness to target appearance changes. Different update frequencies are adopted for the three filters to adapt to severe appearance changes. We perform extensive experiments on two benchmarks: OTB-50 and OTB-100. Quantitative and qualitative evaluations show that the proposed tracking method performs favorably against some state-of-the-art methods – even better than algorithms using complex network model. Furthermore, proposed algorithm runs faster than 20 frame per second (FPS) and hence able to achieve near real-time tracking.

ACS Style

Yuhang Pang; Fan Li; Xiaoya Qiao; Andrew Gilman. Real-time tracking based on deep feature fusion. Multimedia Tools and Applications 2020, 79, 27229 -27255.

AMA Style

Yuhang Pang, Fan Li, Xiaoya Qiao, Andrew Gilman. Real-time tracking based on deep feature fusion. Multimedia Tools and Applications. 2020; 79 (37-38):27229-27255.

Chicago/Turabian Style

Yuhang Pang; Fan Li; Xiaoya Qiao; Andrew Gilman. 2020. "Real-time tracking based on deep feature fusion." Multimedia Tools and Applications 79, no. 37-38: 27229-27255.

Journal article
Published: 08 April 2020 in Neurocomputing
Reads 0
Downloads 0

Deep-learning based image quality assessment (IQA) algorithms usually use the transfer learning method that transfers a pre-trained network for classification task to handle IQA task. Although it can overcome the problem of having insufficient IQA databases to some extent, it cannot distinguish between the important and unimportant deep features for the IQA task, which potentially leads to inaccurate prediction performance. In this paper, we propose a no-reference IQA method based on modelling of deep feature importance. A SE-VGG network is developed by using adaptive transfer learning method. It can suppress the features of local parts of salient objects of images that are not important to the IQA task, and emphasize the features of image distortion and salient objects that are important to IQA task. Moreover, the structure of the SE-VGG is investigated to improve the accuracy of the image quality assessment on a small IQA database. Experiments are conducted to evaluate the performance of the proposed method on various databases, including the LIVE, TID2013, CSIQ, LIVE multiply distorted and LIVE challenge. The results show the proposed method significantly outperforms the state-of-the-art methods. In addition, our method demonstrates a strong generalization ability.

ACS Style

Xiaohan Yang; Fan Li; Hantao Liu. Deep feature importance awareness based no-reference image quality prediction. Neurocomputing 2020, 401, 209 -223.

AMA Style

Xiaohan Yang, Fan Li, Hantao Liu. Deep feature importance awareness based no-reference image quality prediction. Neurocomputing. 2020; 401 ():209-223.

Chicago/Turabian Style

Xiaohan Yang; Fan Li; Hantao Liu. 2020. "Deep feature importance awareness based no-reference image quality prediction." Neurocomputing 401, no. : 209-223.

Journal article
Published: 03 February 2020 in Electronics
Reads 0
Downloads 0

In this paper, we propose a no-reference image quality assessment (NR-IQA) approach towards authentically distorted images, based on expanding proxy labels. In order to distinguish from the human labels, we define the quality score, which is generated by using a traditional NR-IQA algorithm, as “proxy labels”. “Proxy” means that the objective results are obtained by computer after the extraction and assessment of the image features, instead of human judging. To solve the problem of limited image quality assessment (IQA) dataset size, we adopt a cascading transfer-learning method. First, we obtain large numbers of proxy labels which denote the quality score of authentically distorted images by using a traditional no-reference IQA method. Then the deep network is trained by the proxy labels, in order to learn IQA-related knowledge from the amounts of images with their scores. Ultimately, we use fine-tuning to inherit knowledge represented in the trained network. During the procedure, the mapping relationship fits in with human visual perception closer. The experimental results demonstrate that the proposed algorithm shows an outstanding performance as compared with the existing algorithms. On the LIVE In the Wild Image Quality Challenge database and KonIQ-10k database (two standard databases for authentically distorted image quality assessment), the algorithm realized good consistency between human visual perception and the predicted quality score of authentically distorted images.

ACS Style

Xiaodi Guan; Fan Li; Lijun He. Quality Assessment on Authentically Distorted Images by Expanding Proxy Labels. Electronics 2020, 9, 252 .

AMA Style

Xiaodi Guan, Fan Li, Lijun He. Quality Assessment on Authentically Distorted Images by Expanding Proxy Labels. Electronics. 2020; 9 (2):252.

Chicago/Turabian Style

Xiaodi Guan; Fan Li; Lijun He. 2020. "Quality Assessment on Authentically Distorted Images by Expanding Proxy Labels." Electronics 9, no. 2: 252.

Journal article
Published: 31 December 2019 in Entropy
Reads 0
Downloads 0

Image quality assessment (IQA) is a fundamental technology for image applications that can help correct low-quality images during the capture process. The ability to expand distorted images and create human visual system (HVS)-aware labels for training is the key to performing IQA tasks using deep neural networks (DNNs), and image quality is highly sensitive to changes in entropy. Therefore, a new data expansion method based on entropy and guided by saliency and distortion is proposed in this paper. We introduce saliency into a large-scale expansion strategy for the first time. We regionally add distortion to a set of original images to obtain a distorted image database and label the distorted images using entropy. The careful design of the distorted images and the entropy-based labels fully reflects the influences of both saliency and distortion on quality. The expanded database plays an important role in the application of a DNN for IQA. Experimental results on IQA databases demonstrate the effectiveness of the expansion method, and the network’s prediction effect on the IQA databases is found to be improved compared with its predecessor algorithm. Therefore, we conclude that a data expansion approach that fully reflects HVS-aware quality factors is beneficial for IQA. This study presents a novel method for incorporating saliency into IQA, namely, representing it as regional distortion.

ACS Style

Xiaodi Guan; Lijun He; Mengyue Li; Fan Li. Entropy Based Data Expansion Method for Blind Image Quality Assessment. Entropy 2019, 22, 60 .

AMA Style

Xiaodi Guan, Lijun He, Mengyue Li, Fan Li. Entropy Based Data Expansion Method for Blind Image Quality Assessment. Entropy. 2019; 22 (1):60.

Chicago/Turabian Style

Xiaodi Guan; Lijun He; Mengyue Li; Fan Li. 2019. "Entropy Based Data Expansion Method for Blind Image Quality Assessment." Entropy 22, no. 1: 60.

Journal article
Published: 02 September 2019 in IEEE Access
Reads 0
Downloads 0

Blind image quality assessment (BIQA) methods aim to predict quality of images as perceived by humans without access to a reference image. Recently, deep learning methods have gained substantial attention in the research community and have proven useful for BIQA. Although previous study of deep neural networks (DNN) methods is presented, some novelty DNN methods, which are recently proposed, are not summarized for BIQA. In this paper, we provide a survey covering various DNN methods for BIQA. First, we systematically analyze the existing DNN-based quality assessment methods according to the role of DNN. Then, we compare the prediction performance of various DNN methods on the synthetic databases (LIVE, TID2013, CSIQ, LIVE multiply distorted) and authentic databases (LIVE challenge), providing important information that can help understand the underlying properties between different DNN methods for BIQA. Finally, we describe some emerging challenges in designing and training DNN-based BIQA, along with few directions that are worth further investigations in the future.

ACS Style

Xiaohan Yang; Fan Li; Hantao Liu. A Survey of DNN Methods for Blind Image Quality Assessment. IEEE Access 2019, 7, 123788 -123806.

AMA Style

Xiaohan Yang, Fan Li, Hantao Liu. A Survey of DNN Methods for Blind Image Quality Assessment. IEEE Access. 2019; 7 (99):123788-123806.

Chicago/Turabian Style

Xiaohan Yang; Fan Li; Hantao Liu. 2019. "A Survey of DNN Methods for Blind Image Quality Assessment." IEEE Access 7, no. 99: 123788-123806.

Article
Published: 14 June 2019 in Multimedia Tools and Applications
Reads 0
Downloads 0

We consider multiuser video communication over uplink orthogonal frequency-division multiple access (OFDMA) systems. A cross-layer algorithm of joint bit allocation, packet scheduling and wireless resource assignment are proposed to minimize the end-to-end expected video distortion. Video rate adaptation is performed under the wireless resource constraints. The target number of encoding bits for each video packet is obtained to minimize the estimated distortion based on the online content-based rate-distortion function. Due to the inaccuracy of the rate control algorithm in H.265/HEVC encoding, the actual number of bits may differ from the target. Accordingly, the actual encoder distortion may deviate from the estimated distortion. Then, we propose an iterative algorithm to re-assign wireless resources based on the actual number of encoded bits to obtain the final resource allocation policy and packet scheduling decision. Numerical simulation results show that our proposed approach significantly outperforms the baseline algorithms in terms of received video quality.

ACS Style

Fan Li; Taiyu Wang; Pamela C. Cosman. Joint rate adaptation and resource allocation for real-time H.265/HEVC video transmission over uplink OFDMA systems. Multimedia Tools and Applications 2019, 78, 26807 -26831.

AMA Style

Fan Li, Taiyu Wang, Pamela C. Cosman. Joint rate adaptation and resource allocation for real-time H.265/HEVC video transmission over uplink OFDMA systems. Multimedia Tools and Applications. 2019; 78 (18):26807-26831.

Chicago/Turabian Style

Fan Li; Taiyu Wang; Pamela C. Cosman. 2019. "Joint rate adaptation and resource allocation for real-time H.265/HEVC video transmission over uplink OFDMA systems." Multimedia Tools and Applications 78, no. 18: 26807-26831.

Research article
Published: 01 June 2019 in IET Communications
Reads 0
Downloads 0

With the rapid development of dynamic adaptive streaming over HTTP (DASH) services, how to satisfy the requirements of DASH clients has attracted more and more attention. This study was carried out to focus on the quality of experience (QoE) modelling and the model-based cross-layer design which consists of segment adaptation and resource allocation, improving the playback experience of the clients. First, the key factors are investigated, which can affect the clients’ subjective satisfaction. A characteristics and playback information of the segments-based QoE (CPIQ) model is established by employing the curve-fitting method based on a large amount of subjective experimental results of these factors. Then with the CPIQ model as the objective function, a cross-layer design CPIQ model-based joint algorithm of segment request and resource allocation (CJRRA) model is formulated to maximise the total QoE of all the clients subject to the network resource and segment representation constraints. Segment adaptation and resource allocation strategy can be determined jointly by the authors developed low complexity solution. Finally, the simulation results show that the overall performance of CPIQ model and CJRRA significantly outperforms other compared models or algorithms in terms of accuracy, linearity and stability.

ACS Style

Jin Yang; Ruiping Qiao; Ruijie Ma; Fan Li. Cross‐layer design of resource allocation and segment adaptation based on CPIQ model for DASH clients over LTE networks. IET Communications 2019, 13, 1405 -1414.

AMA Style

Jin Yang, Ruiping Qiao, Ruijie Ma, Fan Li. Cross‐layer design of resource allocation and segment adaptation based on CPIQ model for DASH clients over LTE networks. IET Communications. 2019; 13 (10):1405-1414.

Chicago/Turabian Style

Jin Yang; Ruiping Qiao; Ruijie Ma; Fan Li. 2019. "Cross‐layer design of resource allocation and segment adaptation based on CPIQ model for DASH clients over LTE networks." IET Communications 13, no. 10: 1405-1414.

Journal article
Published: 20 November 2018 in IEEE Access
Reads 0
Downloads 0

Flexible mode selection in one CTU, as one of the new technologies in H.265/HEVC video coding, can attain a high compression ratio. However, it may decrease the end-to-end PSNR for video communication over lossy networks. In this paper, we analyze the flexible prediction mode for the CTU in H.265/HEVC and find it causes more error propagation than fixed prediction mode in sub-pixel interpolation. Then, fixed mode based error resilient encoding (FERE) is proposed to decrease the influence of error propagation. Experimental results show that the proposed FERE algorithm improves video quality by up to 5.56dB compared with the baseline algorithms.

ACS Style

Taiyu Wang; Fan Li; Pamela C. Cosman. H.265/HEVC Video Coding Over Lossy Networks: Flexible or Fixed Mode in One CTU? IEEE Access 2018, 6, 71279 -71284.

AMA Style

Taiyu Wang, Fan Li, Pamela C. Cosman. H.265/HEVC Video Coding Over Lossy Networks: Flexible or Fixed Mode in One CTU? IEEE Access. 2018; 6 (99):71279-71284.

Chicago/Turabian Style

Taiyu Wang; Fan Li; Pamela C. Cosman. 2018. "H.265/HEVC Video Coding Over Lossy Networks: Flexible or Fixed Mode in One CTU?" IEEE Access 6, no. 99: 71279-71284.