Jinkyu Kim

Jinkyu Kim

Auto-generated profile, based on publicly available information

Jinkyu Kim

Korea University

Research Focus & Metrics

Academic Career Length

9Years (2018-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
9 October 2018
Verified
Book Chapter
Closed access
PDF file
JKJinkyu Kim
Jinkyu Kim
ARAnna Rohrbach
Anna Rohrbach
TDTrevor Darrell
Trevor Darrell
JCJohn Canny
John Canny
ZAZeynep Akata
Zeynep Akata

14 February 2022 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
RMReza Mahjourian
Reza Mahjourian
JKJinkyu Kim
Jinkyu Kim
YCYuning Chai
Yuning Chai
MTMingxing Tan
Mingxing Tan
BSBen Sapp
Ben Sapp
DADragomir Anguelov
Dragomir Anguelov
We propose Occupancy Flow Fields, a new representation for motion forecasting of multiple agents, an important task in autonomous driving.Our representation is a spatio-temporal grid with each grid cell containing both the probability of the cell being occupied by any agent, and a two-dimensional flow vector representing the direction and magnitude of the motion in that cell. Our method successfully mitigates shortcomings of the two most commonly-used representations for motion forecasting: trajectory sets and occupancy grids. Although occupancy grids efficiently represent the probabilistic location of many agents jointly, they do not capture agent motion and lose the agent identities. To this end, we propose a deep learning architecture that generates Occupancy Flow Fields with the help of a new flow trace loss that establishes consistency between the occupancy and flow predictions. We demonstrate the effectiveness of our approach using three metrics on occupancy prediction, motion estimation, and agent ID recovery. In addition, we introduce the problem of predicting speculative agents, which are currently-occluded agents that may appear in the future through dis-occlusion or by entering the field of view. We report experimental results on a large in-house autonomous driving dataset and the public INTERACTION dataset, and show that our model outperforms state-of-the-art models.

24 October 2022
Verified
Book Chapter
Closed access
PDF file
SLSeunghyun Lee
Seunghyun Lee
GOGyeongrok Oh
Gyeongrok Oh
WBWonmin Byeon
Wonmin Byeon
CKChanyoung Kim
Chanyoung Kim
WRWon Jeong Ryoo
Won Jeong Ryoo
SKSangpil Kim
Sangpil Kim

1 March 2020
Verified
Conference Paper
Closed access
YXYe Xia
Ye Xia
JKJinkyu Kim
Jinkyu Kim
JCJohn Canny
John Canny
KZKarl Zipser
Karl Zipser
TCTeresa Canas-Bajo
Teresa Canas-Bajo
DWDavid Whitney
David Whitney
Inspired by human vision, we propose a new periphery-fovea multi-resolution driving model that predicts vehicle speed from dash camera videos. The peripheral vision module of the model processes the full video frames in low resolution with large receptive fields. Its foveal vision module selects sub-regions and uses high-resolution input from those regions to improve its driving performance. We train the fovea selection module with supervision from driver gaze. We show that adding high-resolution input from predicted human driver gaze locations significantly improves the driving accuracy of the model. Our periphery-fovea multi-resolution model outperforms a uni-resolution periphery-only model that has the same amount of floating-point operations. More importantly, we demonstrate that our driving model achieves a significantly higher performance gain in pedestrian-involved critical situations than in other non-critical situations. Our code is publicly available at https://github.com/pascalxia/periphery_fovea_driving.

1 October 2023
Verified
Conference Paper
Open access
YJYujin Jeong
Yujin Jeong
WRWonjeong Ryoo
Wonjeong Ryoo
SLSeunghyun Lee
Seunghyun Lee
DSDabin Seo
Dabin Seo
WBWonmin Byeon
Wonmin Byeon
JKJinkyu Kim
Jinkyu Kim
In recent years, video generation has become a prominent generative tool and has drawn significant attention. However, there is little consideration in audio-to-video generation, though audio contains unique qualities like temporal semantics and magnitude. Hence, we propose The Power of Sound (TPoS) model to incorporate audio input that includes both changeable temporal semantics and magnitude. To generate video frames, TPoS utilizes a latent stable diffusion model with textual semantic information, which is then guided by the sequential audio embedding from our pretrained Audio Encoder. As a result, this method produces audio reactive video contents. We demonstrate the effectiveness of TPoS across various tasks and compare its results with current state-of-the-art techniques in the field of audio-to-video generation. More examples are available at https://ku-vai.github.io/TPoS/

23 November 2021 in Applied AI Letters
Verified
Journal Article
Open access
PDF file
JKJinkyu Kim
Jinkyu Kim
ARAnna Rohrbach
Anna Rohrbach
ZAZeynep Akata
Zeynep Akata
SMSuhong Moon
Suhong Moon
TMTeruhisa Misu
Teruhisa Misu
JCJohn Canny
John Canny
Humans learn to drive through both practice and theory, for example, by studying the rules, while most self‐driving systems are limited to the former. Being able to incorporate human knowledge of typical causal driving behavior should benefit autonomous systems. We propose a new approach that learns vehicle control with the help of human advice. Specifically, our system learns to summarize its visual observations in natural language, predict an appropriate action response (eg, “I see a pedestrian crossing, so I stop”), and predict the controls, accordingly. Moreover, to enhance the interpretability of our system, we introduce a fine‐grained attention mechanism that relies on semantic segmentation and object‐centric RoI pooling. We show that our approach of training the autonomous system with human advice, grounded in a rich semantic representation, matches or outperforms prior work in terms of control prediction and explanation generation. Our approach also results in more interpretable visual explanations by visualizing object‐centric attention maps. We evaluate our approach on a novel driving dataset with ground‐truth human explanations, the Berkeley DeepDrive eXplanation (BDD‐X) dataset.