Seungryul Baek

Dr. Seungryul Baek

Department of Computer Science (CS) and the Artificial Intelligence Graduate School (AIGS), Ulsan National Institute of Science and Technology, Ulsan 44919, Korea

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
3 November 2022
Verified
Book Chapter
Closed access
PDF file
JCJunuk Cha
Junuk Cha
MSMuhammad Saqlain
Muhammad Saqlain
GKGeonU Kim
GeonU Kim
MSMingyu Shin
Mingyu Shin
Seungryul BaekSeungryul Baek
Seungryul Baek

21 June 2022 in IEEE Access
Verified
Journal Article
Open access
JCJunuk Cha
Junuk Cha
MSMuhammad Saqlain
Muhammad Saqlain
DKDonguk Kim
Donguk Kim
SLSeungeun Lee
Seungeun Lee
SLSeongyeong Lee
Seongyeong Lee
Seungryul BaekSeungryul Baek
Seungryul Baek
Skeleton-based human action recognition has attracted significant interest due to its simplicity and good accuracy. Diverse end-to-end trainable frameworks based on skeletal representation have been proposed so far to map the representation to human action classes better. Most skeleton-based human action recognition approaches are based on the skeletons, which are heuristically pre-defined by the commercial sensors. Nevertheless, it is not confirmed whether the sensor-captured skeletons is the best representation of human bodies for the action recognition task, while in general, the dedicated representation is required for achieving the successful performance on subsequent tasks such as action recognition. In this paper, we try to deal with the issue by explicitly learning the skeletal representation in the context of the human action recognition task. We start our investigation by reconstructing 3D meshes of the human bodies from RGB videos. Then we involve the transformer architecture to sample the most informative skeletal representation from reconstructed 3D meshes, considering the inner and inter structural relationship of 3D meshes and sensor-captured skeletons. Experimental results on challenging human action recognition benchmarks (i.e., SYSU and UTD-MHAD datasets) have shown the superiority of our skeletal representation compared to the sensor-captured skeletons for the action recognition task.

18 October 2021 in Applied Sciences
Verified
Journal Article
Open access
PDF file
JCJunuk Cha
Junuk Cha
MSMuhammad Saqlain
Muhammad Saqlain
CLChanghwa Lee
Changhwa Lee
SLSeongyeong Lee
Seongyeong Lee
SLSeungeun Lee
Seungeun Lee
Seungryul BaekSeungryul Baek
Seungryul Baek
Three-dimensional human pose and shape estimation is an important problem in the computer vision community, with numerous applications such as augmented reality, virtual reality, human computer interaction, and so on. However, training accurate 3D human pose and shape estimators based on deep learning approaches requires a large number of images and corresponding 3D ground-truth pose pairs, which are costly to collect. To relieve this constraint, various types of weakly or self-supervised pose estimation approaches have been proposed. Nevertheless, these methods still involve supervision signals, which require effort to collect, such as unpaired large-scale 3D ground truth data, a small subset of 3D labeled data, video priors, and so on. Often, they require installing equipment such as a calibrated multi-camera system to acquire strong multi-view priors. In this paper, we propose a self-supervised learning framework for 3D human pose and shape estimation that does not require other forms of supervision signals while using only single 2D images. Our framework inputs single 2D images, estimates human 3D meshes in the intermediate layers, and is trained to solve four types of self-supervision tasks (i.e., three image manipulation tasks and one neural rendering task) whose ground-truths are all based on the single 2D images themselves. Through experiments, we demonstrate the effectiveness of our approach on 3D human pose benchmark datasets (i.e., Human3.6M, 3DPW, and LSP), where we present the new state-of-the-art among weakly/self-supervised methods.

14 February 2022 in Sensors
Verified
Journal Article
Open access
PDF file
MSMuhammad Saqlain
Muhammad Saqlain
DKdonguk kim
donguk kim
JCJunuk Cha
Junuk Cha
CLChanghwa Lee
Changhwa Lee
SLSeongyeong Lee
Seongyeong Lee
Seungryul BaekSeungryul Baek
Seungryul Baek
Group activity recognition is a prime research topic in video understanding and has many practical applications, such as crowd behavior monitoring, video surveillance, etc. To understand the multi-person/group action, the model should not only identify the individual person’s action in the context but also describe their collective activity. A lot of previous works adopt skeleton-based approaches with graph convolutional networks for group activity recognition. However, these approaches are subject to limitation in scalability, robustness, and interoperability. In this paper, we propose 3DMesh-GAR, a novel approach to 3D human body Mesh-based Group Activity Recognition, which relies on a body center heatmap, camera map, and mesh parameter map instead of the complex and noisy 3D skeleton of each person of the input frames. We adopt a 3D mesh creation method, which is conceptually simple, single-stage, and bounding box free, and is able to handle highly occluded and multi-person scenes without any additional computational cost. We implement 3DMesh-GAR on a standard group activity dataset: the Collective Activity Dataset, and achieve state-of-the-art performance for group activity recognition.

11 April 2025 in Proceedings of the AAAI Conference on Artificial Intelligence
Verified
Journal Article
Open access
PDF file
EIElkhan Ismayilzada
Elkhan Ismayilzada
KSKhalequzzaman Chowdhury Sayem
Khalequzzaman Chowdhury Sayem
YTYihalem Yimolal Tiruneh
Yihalem Yimolal Tiruneh
MCMubarrat Tajoar Chowdhury
Mubarrat Tajoar Chowdhury
MBMuhammadjon Boboev
Muhammadjon Boboev
Seungryul BaekSeungryul Baek
Seungryul Baek
Significant advancements have been achieved in the realm of understanding poses and interactions of two hands manipulating an object. The emergence of augmented reality (AR) and virtual reality (VR) technologies has heightened the demand for real-time performance in these applications. However, current state-of-the-art models often exhibit promising results at the expense of substantial computational overhead. In this paper, we present a query-optimized real-time Transformer (QORT-Former), the first Transformer-based real-time framework for 3D pose estimation of two hands and an object. We first limit the number of queries and decoders to meet the efficiency requirement. Given limited number of queries and decoders, we propose to optimize queries which are taken as input to the Transformer decoder, to secure better accuracy: (1) we propose to divide queries into three types (a left hand query, a right hand query and an object query) and enhance query features (2) by using the contact information between hands and an object and (3) by using three-step update of enhanced image and query features with respect to one another. With proposed methods, we achieved real-time pose estimation performance using just 108 queries and 1 decoder (53.5 FPS on an RTX 3090TI GPU). Surpassing state-of-the-art results on the H2O dataset by 17.6% (left hand), 22.8% (right hand), and 27.2% (object), as well as on the FPHA dataset by 5.3% (right hand) and 10.4% (object), our method excels in accuracy. Additionally, it sets the state-of-the-art in interaction recognition, maintaining real-time efficiency with an off-the-shelf action recognition module.

9 November 2025 in arXiv
Verified
Preprint
Open access
PDF file
SASanskar Amgain
Sanskar Amgain
PSPrashant Shrestha
Prashant Shrestha
BKBidur Khanal
Bidur Khanal
ADAlina Devkota
Alina Devkota
YSYash Raj Shrestha
Yash Raj Shrestha
BBBinod Bhattarai
Binod Bhattarai
Federated learning on clients with noisy labels is a challenging problem, as such clients can infiltrate the global model, impacting the overall generalizability of the system. Existing methods proposed to handle noisy clients assume that a sufficient number of clients with clean labels are available, which can be leveraged to learn a robust global model while dampening the impact of noisy clients. This assumption fails when a high number of heterogeneous clients contain noisy labels, making the existing approaches ineffective. In such scenarios, it is important to locally regularize the clients before communication with the global model, to ensure the global model isn't corrupted by noisy clients. While pre-trained self-supervised models can be effective for local regularization, existing centralized approaches relying on pretrained initialization are impractical in a federated setting due to the potentially large size of these models, which increases communication costs. In that line, we propose a regularization objective for client models that decouples the pre-trained and classification models by enforcing similarity between close data points within the client. We leverage the representation space of a self-supervised pretrained model to evaluate the closeness among examples. This regularization, when applied with the standard objective function for the downstream task in standard noisy federated settings, significantly improves performance, outperforming existing state-of-the-art federated methods in multiple computer vision and medical image classification benchmarks. Unlike other techniques that rely on self-supervised pretrained initialization, our method does not require the pretrained model and classifier backbone to share the same architecture, making it architecture-agnostic.