Joon-Young Lee

Joon-Young Lee

Auto-generated profile, based on publicly available information

Joon-Young Lee

Adobe Research

Research Focus & Metrics

Academic Career Length

16Years (2011-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 June 2015
Verified
Conference Paper
Closed access
DYDonggeun Yoo
Donggeun Yoo
SPSunggyun Park
Sunggyun Park
JLJoon-Young Lee
Joon-Young Lee
In So KweonIn So Kweon
In So Kweon
Compared to image representation based on low-level local descriptors, deep neural activations of Convolutional Neural Networks (CNNs) are richer in mid-level representation, but poorer in geometric invariance properties. In this paper, we present a straightforward framework for better image representation by combining the two approaches. To take advantages of both representations, we extract a fair amount of multi-scale dense local activations from a pre-trained CNN. We then aggregate the activations by Fisher kernel framework, which has been modified with a simple scale-wise normalization essential to make it suitable for CNN activations. Our representation demonstrates new state-of-the-art performances on three public datasets: 80.78% (Acc.) on MIT Indoor 67, 83.20% (mAP) on PASCAL VOC 2007 and 91.28% (Acc.) on Oxford 102 Flowers. The results suggest that our proposal can be used as a primary image representation for better performances in wide visual recognition tasks.

2 February 2018 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
GCGyeongmin Choe
Gyeongmin Choe
Seong-heum KimSeong-heum Kim
Seong-heum Kim
SISunghoon Im
Sunghoon Im
JLJoon-Young Lee
Joon-Young Lee
SNSrinivasa G. Narasimhan
Srinivasa G. Narasimhan
IKIn So Kweon
In So Kweon
In this letter, we present a data-driven method for scene parsing of road scenes to utilize single-channel near-infrared (NIR) images. To overcome the lack of data problem in non-RGB spectrum, we define a new color space and decompose the task of deep scene parsing into two subtasks with two separate CNN architectures for chromaticity channels and semantic masks. For chromaticity estimation, we build a spatially-aligned RGB-NIR image database (40k urban scenes) to infer color information from RGB-NIR spectrum learning process and leverage existing scene parsing networks trained over already available RGB masks. From our database, we sample key frames and manually annotate them (4k ground truth masks) to finetune the network into the proposed color space. Hence, the key contribution of this work is to replace multispectral scene parsing methods with a simple yet effective approach using single NIR images. The benefits of using our algorithm and dataset are confirmed in the qualitative and quantitative experiments.

11 December 2019 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
DKDahun Kim
Dahun Kim
SWSanghyun Woo
Sanghyun Woo
JLJoon-Young Lee
Joon-Young Lee
In So KweonIn So Kweon
In So Kweon
Video inpainting aims to fill in spatio-temporal holes in videos with plausible content. Despite tremendous progress on deep learning-based inpainting of a single image, it is still challenging to extend these methods to video domain due to the additional time dimension. In this paper, we propose a recurrent temporal aggregation framework for fast deep video inpainting. In particular, we construct an encoder-decoder model, where the encoder takes multiple source frames which can provide visible pixels revealed from the scene dynamics. These hints are aggregated and fed into the decoder. We apply a recurrent feedback in an auto-regressive manner to enforce temporal consistency in the video results. We propose two architectural designs based on this framework. Our first model is a blind video decaptioning network (BVDNet) that is designed to automatically remove and inpaint text overlays in videos without any mask information. Our BVDNet wins the first place in the ECCV Chalearn 2018 LAP Inpainting Competition Track 2: Video Decaptioning. Second, we propose a network for more general video inpainting (VINet) to deal with more arbitrary and larger holes. Video results demonstrate the advantage of our framework compared to state-of-the-art methods both qualitatively and quantitatively.

11 June 2018 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
ISInwook Shim
Inwook Shim
TOTae-Hyun Oh
Tae-Hyun Oh
JLJoon-Young Lee
Joon-Young Lee
JCJinwook Choi
Jinwook Choi
DCDong-Geol Choia
Dong-Geol Choia
IKIn So Kweon
In So Kweon
We introduce a novel method to automatically adjust camera exposure for image processing and computer vision applications on mobile robot platforms. Because most image processing algorithms rely heavily on low-level image features that are based mainly on local gradient information, we consider that gradient quantity can determine the proper exposure level, allowing a camera to capture the important image features in a manner robust to illumination conditions. We then extend this concept to a multi-camera system and present a new control algorithm to achieve both brightness consistency between adjacent cameras and a proper exposure level for each camera. We implement our prototype system with off-theshelf machine-vision cameras and demonstrate the effectiveness of the proposed algorithms on practical applications, including pedestrian detection, visual odometry, surround-view imaging, panoramic imaging and stereo matching.

13 July 2020 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
SOSeoung Wug Oh
Seoung Wug Oh
JLJoon-Young Lee
Joon-Young Lee
NXNing Xu
Ning Xu
SKSeon Joo Kim
Seon Joo Kim
We propose a novel and unified solution for user-guided video object segmentation tasks. In this work, we consider two scenarios of user-guided segmentation: semi-supervised and interactive segmentation. Due to the nature of the problem, available cues -- video frame(s) with object masks (or scribbles) -- become richer with the intermediate predictions (or additional user inputs). However, the existing methods make it impossible to fully exploit this rich source of information. We resolve the issue by leveraging memory networks and learning to read relevant information from all available sources. In the semi-supervised scenario, the previous frames with object masks form an external memory, and the current frame as the query is segmented using the information in the memory. Similarly, to work with user interactions, the frames that are given user inputs form the memory that guides segmentation. Internally, the query and the memory are densely matched in the feature space, covering all the space-time pixel locations in a feed-forward fashion. The abundant use of the guidance information allows us to better handle challenges such as appearance changes and occlusions. We validate our method on the latest benchmark sets and achieve state-of-the-art performance along with a fast runtime.

8 May 2017 in The Visual Computer
Verified
Journal Article
Closed access
PDF file
JJJinwoong Jung
Jinwoong Jung
BKBeomseok Kim
Beomseok Kim
JLJoon-Young Lee
Joon-Young Lee
BKByungmoon Kim
Byungmoon Kim
SLSeungyong Lee
Seungyong Lee