Kai Lv

Dr. Kai Lv

orcid

Research Focus & Metrics

Academic Career Length

9Years (2018-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
17 November 2020 in IEEE Transactions on Multimedia
Verified
Journal Article
Open access
KLKai Lv
Kai Lv
HSHao Sheng
Hao Sheng
ZXZhang Xiong
Zhang Xiong
WLWei Li
Wei Li
LZLiang Zheng
Liang Zheng
We consider the task of driver gaze prediction: estimating where the location of the focus of a driver should be, based on a raw video of the outside environment. In practice, we output a probability map that gives the normalized probability of each point in a given scene being the object of the driver attention. Most existing methods ( i.e. , Coarse-to-Fine and Multi-branch ) take an image or a video as input and directly output the fixation map. While successful, these methods can often produce highly scattered predictions, rendering them unreliable for real-world usage. Motivated by this observation, we propose the reinforced attention (RA) model as a regulatory mechanism to increase prediction density. Our method is built directly on top of existing methods, making it complementary to current approaches. Specifically, we first use Multi-branch to obtain an initial fixation map. Then, RA is trained using deep reinforcement learning to learn a location prediction policy, producing a reinforced attention. Finally, in order to obtain the final gaze prediction result, we combine the fixation map and the reinforced attention by a mask-guided multiplication. Experimental results show that our framework improves the accuracy of gaze prediction, and provides state-of-the-art performance on the DR(eye)VE dataset.

9 February 2023 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
SWShuo Wang
Shuo Wang
ZWZhihao Wu
Zhihao Wu
XHXiaobo Hu
Xiaobo Hu
YLYoufang Lin
Youfang Lin
KLKai Lv
Kai Lv
Target visual navigation aims at controlling the agent to find a target object based on a monocular visual RGB image in each step. It is crucial for the agent to adapt to new environments. As target visual navigation is a complex task, understanding the behavior of the agent is beneficial for analyzing the reasons for failure. This work focuses on improving the readability and success rate of navigation policies. In this paper, we propose a framework named Skill-based Hierarchical Reinforcement Learning (SHRL) for target visual navigation. SHRL contains a high-level policy and three low-level skills. The high-level policy accomplishes the task by utilizing or stopping low-level skills at each step. Low-level skills are designed to separately solve three sub-tasks, i.e., Search, Adjustment , and Exploration . In addition, we propose an Abstract Representation and two penalty items to feed robust features to the high-level policy. Abstract Representation is designed to focus on selecting low-level skills rather than the details of navigation. Experimental results in the artificial environment AI2-Thor indicate that the proposed method outperforms state-of-the-art by a large margin in unseen indoor environments. Moreover, we also provide case studies to illustrate the advantages of SHRL.

19 March 2020 in IEEE Transactions on Image Processing
Verified
Journal Article
Open access
KLKai Lv
Kai Lv
HSHao Sheng
Hao Sheng
ZXZhang Xiong
Zhang Xiong
WLWei Li
Wei Li
LZLiang Zheng
Liang Zheng
In this paper, we focus on the problem of novel view synthesis for vehicles. Some previous works solve the problem of novel view synthesis in a controlled 3D environment by exploiting additional 3D details (i.e., camera viewpoints and underlying 3D models). However, in real scenarios, the 3D details are difficult to obtain. In this case, we find that introducing vehicle pose to represent the views of vehicles is an alternative paradigm to solve the lack of 3D details. In novel view synthesis, preserving local details is one of the most challenging problems. To address this problem, we propose a perspective-aware generative model (PAGM). We are motivated by the prior that vehicles are made of quadrilateral planes. Preserving these rigid planes during image generation ensures that image details are kept. To this end, a classic image transformation method is leveraged, i.e., perspective transformation. In our GAN-based system, the perspective transformation is applied to the encoder feature maps, and the resulting maps are regarded as new conditions for the decoder. This strategy preserves the quadrilateral planes all the way through the network, thus shuttling the texture details from the input image to the generated image. In the experiments, we show that PAGM can generate high-quality vehicle images with fine details. Quantitatively, our method is superior to several competing approaches employing either GAN or the perspective transformation. Code is available at: https://github.com/ilvkai/view-synthesis-for-vehicles.

5 December 2022 in IEEE Transactions on Vehicular Technology
Verified
Journal Article
Closed access
HZHengrui Zhang
Hengrui Zhang
YLYoufang Lin
Youfang Lin
SHSheng Han
Sheng Han
KLKai Lv
Kai Lv
Urban autonomous driving is a difficult task because of its complex road scenarios and the interaction between multiple vehicles. Autonomous vehicles need to balance multiple objectives in these complex scenarios, e.g. , safety and speed. Traditional reinforcement learning methods deal with the multi-objective problem by optimizing agents with a single objective reward. However, these methods are sensitive to the reward scale and require huge experiments to design reward weights. In this paper, we propose the Lexicographical Proximal Policy Optimization algorithm (LPPO), which can express people's preference relationship through the lexicographical ordering between objectives. The proposed method has two main advantages. On the one hand, LPPO has a smaller parameter adjustment space, which makes it easy to find the optimal solution that satisfies the actual problem preference. On the other hand, the proposed method is less affected by the reward scale and easy to deploy in various driving scenarios. We evaluate our algorithm in two driving simulation environments, and the results show that the proposed method has better performance in urban driving tasks than previous reinforcement learning algorithms. In addition, we illustrate that the proposed method has better stability even if the reward scale changes.

30 June 2023 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
XHXiaobo Hu
Xiaobo Hu
YLYoufang Lin
Youfang Lin
SWShuo Wang
Shuo Wang
ZWZhihao Wu
Zhihao Wu
KLKai Lv
Kai Lv
Object visual navigation aims to steer an agent toward a target object based on visual observations. It is highly desirable to reasonably perceive the environment and accurately control the agent. In the navigation task, we introduce an Agent-Centric Relation Graph (ACRG) for learning the visual representation based on the relationships in the environment. ACRG is a highly effective structure that consists of two relationships, i.e., the horizontal relationship among objects and the distance relationship between the agent and objects . On the one hand, we design the Object Horizontal Relationship Graph (OHRG) that stores the relative horizontal location among objects. On the other hand, we propose the Agent-Target Distance Relationship Graph (ATDRG) that enables the agent to perceive the distance between the target and objects. For ATDRG, we utilize image depth to obtain the target distance and imply the vertical location to capture the distance relationship among objects in the vertical direction. With the above graphs, the agent can perceive the environment and output navigation actions. Experimental results in the artificial environment AI2-THOR demonstrate that ACRG significantly outperforms other state-of-the-art methods in unseen testing environments.

10 August 2020 in IEEE Internet of Things Journal
Verified
Journal Article
Closed access
HSHao Sheng
Hao Sheng
KLKai Lv
Kai Lv
YLYang Liu
Yang Liu
WKWei Ke
Wei Ke
WLWeifeng Lyu
Weifeng Lyu
WLWei Li
Wei Li
Vehicle re-identification, aiming to identify vehicles across images, has drawn a lot attention and has made significant achievements recent years. However, vehicle re-identification still remains a challenging task because of severe appearance changes of a vehicle due to different orientations. In practice, the result of re-identification is greatly influenced by the pose of vehicles and we call this influence as “pose-barrier”. In this paper, we propose to use pose-invariant features and pose-discriminative features to address the pose-barrier. Pose-invariant features are expert in exploring the overall characteristic of the appearance. Meanwhile, pose-discriminative features are prior to distinguish the difference between local details. The framework of our method can be divided into three steps: orientation estimation, pose-invariant feature learning and pose discriminative feature learning, so-called pose-based complementary features (PCF). The pose-discriminative feature is trained together with a orientation discriminator which tries to judge the orientation of input images. On the other hand, adversarial loss is applied in a pose to train a pose-discriminative feature which does not contain orientation information. By combining the two features, the method can alleviate the influence of pose-barrier. Extensive experiments on a series of challenging databases demonstrate that the proposed framework outperforms the state-of-the-art results on vehicle re-identification dataset: VeRi-776.