Seong-Ho Choi

Seong-Ho Choi

Auto-generated profile, based on publicly available information

Seong-Ho Choi

Seoul National University

Research Focus & Metrics

Academic Career Length

10Years (2017-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 August 2017
Verified
Conference Paper
Closed access
KKKyung-Min Kim
Kyung-Min Kim
MHMin-Oh Heo
Min-Oh Heo
SCSeong-Ho Choi
Seong-Ho Choi
BZByoung-Tak Zhang
Byoung-Tak Zhang
Question-answering (QA) on video contents is a significant challenge for achieving human-level intelligence as it involves both vision and language in real-world settings. Here we demonstrate the possibility of an AI agent performing video story QA by learning from a large amount of cartoon videos. We develop a video-story learning model, i.e. Deep Embedded Memory Networks (DEMN), to reconstruct stories from a joint scene-dialogue video stream using a latent embedding space of observed data. The video stories are stored in a long-term memory component. For a given question, an LSTM-based attention model uses the long-term memory to recall the best question-story-answer triplet by focusing on specific words containing key information. We trained the DEMN on a novel QA dataset of children’s cartoon video series, Pororo. The dataset contains 16,066 scene-dialogue pairs of 20.5-hour videos, 27,328 fine-grained sentences for scene description, and 8,913 story-related QA pairs. Our experimental results show that the DEMN outperforms other QA models. This is mainly due to 1) the reconstruction of video stories in a scene-dialogue combined form that utilize the latent embedding and 2) attention. DEMN also achieved state-of-the-art results on the MovieQA benchmark.

7 October 2018
Verified
Book Chapter
Closed access
PDF file
KKKyung-Min Kim
Kyung-Min Kim
SCSeong-Ho Choi
Seong-Ho Choi
JKJin-Hwa Kim
Jin-Hwa Kim
BZByoung-Tak Zhang
Byoung-Tak Zhang

18 May 2021 in Proceedings of the AAAI Conference on Artificial Intelligence
Verified
Journal Article
Open access
PDF file
SCSeongho Choi
Seongho Choi
KOKyoung-Woon On
Kyoung-Woon On
YHYu-Jung Heo
Yu-Jung Heo
ASAhjeong Seo
Ahjeong Seo
YJYouwon Jang
Youwon Jang
BZByoung-Tak Zhang
Byoung-Tak Zhang
Despite recent progress on computer vision and natural language processing, developing a machine that can understand video story is still hard to achieve due to the intrinsic difficulty of video story. Moreover, researches on how to evaluate the degree of video understanding based on human cognitive process have not progressed as yet. In this paper, we propose a novel video question answering (Video QA) task, DramaQA, for a comprehensive understanding of the video story. The DramaQA focuses on two perspectives: 1) Hierarchical QAs as an evaluation metric based on the cognitive developmental stages of human intelligence. 2) Character-centered video annotations to model local coherence of the story. Our dataset is built upon the TV drama "Another Miss Oh" and it contains 17,983 QA pairs from 23,928 various length video clips, with each QA pair belonging to one of four difficulty levels. We provide 217,308 annotated images with rich character-centered annotations, including visual bounding boxes, behaviors and emotions of main characters, and coreference resolved scripts. Additionally, we suggest Multi-level Context Matching model which hierarchically understands character-centered representations of video to answer questions. We release our dataset and model publicly for research purposes, and we expect our work to provide a new perspective on video story understanding research.

17 October 2023 in AI Magazine
Verified
Journal Article
Open access
MLMinsu Lee
Minsu Lee
YHYu‐Jung Heo
Yu‐Jung Heo
SCSeongho Choi
Seongho Choi
WCWoo Suk Choi
Woo Suk Choi
BZByoung-Tak Zhang
Byoung-Tak Zhang