Synthesizing high dynamic range (HDR) images from multiple low-dynamic range (LDR) exposures in dynamic scenes is challenging. There are two major problems caused by the large motions of foreground objects. One is the severe misalignment among the LDR images. The other is the missing content due to the over-/under-saturated regions caused by the moving objects, which may not be easily compensated for by the multiple LDR exposures. Thus, it requires the HDR generation model to be able to properly fuse the LDR images and restore the missing details without introducing artifacts. To address these two problems, we propose in this paper a novel GAN-based model, HDR-GAN, for synthesizing HDR images from multi-exposed LDR images. To our best knowledge, this work is the first GAN-based approach for fusing multi-exposed LDR images for HDR reconstruction. By incorporating adversarial learning, our method is able to produce faithful information in the regions with missing content. In addition, we also propose a novel generator network, with a reference-based residual merging block for aligning large object motions in the feature domain, and a deep HDR supervision scheme for eliminating artifacts of the reconstructed HDR images. Experimental results demonstrate that our model achieves state-of-the-art reconstruction performance over the prior HDR methods on diverse scenes.
A knowledge graph (KG), also known as a knowledge base, is a particular kind of network structure in which the node indicates entity and the edge represent relation. However, with the explosion of network volume, the problem of data sparsity that causes large-scale KG systems to calculate and manage difficultly has become more significant. For alleviating the issue, knowledge graph embedding is proposed to embed entities and relations in a KG to a low-, dense and continuous feature space, and endow the yield model with abilities of knowledge inference and fusion. In recent years, many researchers have poured much attention in this approach, and we will systematically introduce the existing state-of-the-art approaches and a variety of applications that benefit from these methods in this paper. In addition, we discuss future prospects for the development of techniques and application trends. Specifically, we first introduce the embedding models that only leverage the information of observed triplets in the KG. We illustrate the overall framework and specific idea and compare the advantages and disadvantages of such approaches. Next, we introduce the advanced models that utilize additional semantic information to improve the performance of the original methods. We divide the additional information into two categories, including textual descriptions and relation paths. The extension approaches in each category are described, following the same classification criteria as those defined for the triplet fact-based models. We then describe two experiments for comparing the performance of listed methods and mention some broader domain tasks such as question answering, recommender systems, and so forth. Finally, we collect several hurdles that need to be overcome and provide a few future research directions for knowledge graph embedding.
2 November 2021 in Machine Vision and Applications
The recent advances of compressive sensing (CS) have witnessed a great potential of efficient compressive data gathering (CDG) in wireless sensor network systems (WSNSs). However, most existing work on CDG mainly focuses on multihop relaying strategies to improve the performance of data gathering. In this paper, we propose a mobile CDG scheme including a random walk-based algorithm and a kernel-based method for sparsifying sensory data from irregular deployments. The proposed scheme allows a mobile collector to harvest data by sequentially visiting a number of nodes along a random path. More importantly, toward building the gap between CS and machine learning theories, we explore a theoretical foundation for understanding the feasibility of the proposed scheme. We prove that the CS matrices, constructed from the proposed random walk algorithm combined with a kernel-based sparsity basis, satisfy the restricted isometry property. Particularly, we also show that m = O(klog(n/k)) measurements collected by a mobile collector are sufficient to recover a k-sparse signal and t = O(klog(n/k)) steps are required to collect these measurements in a network with n nodes. Finally, we also present extensive numerical results to validate the effectiveness of the proposed scheme by evaluating the performance in terms of energy consumption and the impact of packet losses. The numerical results demonstrate that the proposed scheme is able to not only significantly reduce communication cost but also combat unreliable wireless links under various packet losses compared to the stateof-the-art schemes, which provides an efficient alternative to data relaying approaches for CDG in WSNS.
30 August 2018 in IEEE Transactions on Intelligent Transportation Systems
In intelligent transportation systems, there are many tasks that rely on the detection of road congestion, such as traffic signal scheduling and traffic accident detection. As traditional methods for traffic congestion detection are difficult to use, expensive, and may cause damage to the road surface, this paper presents a method for road congestion detection that is based on multidimensional visual features and a convolutional neural network (CNN). This method first detects the density of foreground objects by using a gray-level co-occurrence matrix; second, the speed of moving objects is detected by using the Lucas-Kanade optical flow with pyramid implementation. Third, a Gaussian mixture model is used to model the background, and the CNN is then used to accurately detect the final foreground from the candidate foregrounds. Finally, the proposed method performs road congestion detection in terms of a multidimensional feature space, including traffic density, traffic velocity, road occupancy, and traffic flow. Furthermore, we propose an information entropy method using a histogram of optical flow to enhance the accuracy and reliability of road congestion detection. Simulation results via quantitative and qualitative assessment indicate that the proposed method is able to significantly outperform the state-of-the-art road-traffic congestion detection methods due to the fusion of multidimensional features using the CNN.