Kihyuk Sohn

Kihyuk Sohn

Auto-generated profile, based on publicly available information

Kihyuk Sohn

Google research

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
2 November 2024
Verified
Book Chapter
Closed access
PDF file
AGAgrim Gupta
Agrim Gupta
LYLijun Yu
Lijun Yu
KSKihyuk Sohn
Kihyuk Sohn
XGXiuye Gu
Xiuye Gu
MHMeera Hahn
Meera Hahn
JLJosé Lezama
José Lezama

25 November 2020 in arXiv
Verified
Preprint
Open access
PDF file
KSKihyuk Sohn
Kihyuk Sohn
DBDavid Berthelot
David Berthelot
CLChun-Liang Li
Chun-Liang Li
ZZZizhao Zhang
Zizhao Zhang
NCNicholas Carlini
Nicholas Carlini
CRColin Raffel
Colin Raffel
Semi-supervised learning (SSL) provides an effective means of leveraging unlabeled data to improve a model's performance. In this paper, we demonstrate the power of a simple combination of two common SSL methods: consistency regularization and pseudo-labeling. Our algorithm, FixMatch, first generates pseudo-labels using the model's predictions on weakly-augmented unlabeled images. For a given image, the pseudo-label is only retained if the model produces a high-confidence prediction. The model is then trained to predict the pseudo-label when fed a strongly-augmented version of the same image. Despite its simplicity, we show that FixMatch achieves state-of-the-art performance across a variety of standard semi-supervised learning benchmarks, including 94.93% accuracy on CIFAR-10 with 250 labels and 88.61% accuracy with 40 -- just 4 labels per class. Since FixMatch bears many similarities to existing SSL methods that achieve worse performance, we carry out an extensive ablation study to tease apart the experimental factors that are most important to FixMatch's success. We make our code available at https://github.com/google-research/fixmatch.

14 February 2020 in arXiv
Verified
Preprint
Open access
PDF file
DBDavid Berthelot
David Berthelot
NCNicholas Carlini
Nicholas Carlini
ECEkin D. Cubuk
Ekin D. Cubuk
AKAlex Kurakin
Alex Kurakin
KSKihyuk Sohn
Kihyuk Sohn
CRColin Raffel
Colin Raffel
We improve the recently-proposed "MixMatch" semi-supervised learning algorithm by introducing two new techniques: distribution alignment and augmentation anchoring. Distribution alignment encourages the marginal distribution of predictions on unlabeled data to be close to the marginal distribution of ground-truth labels. Augmentation anchoring feeds multiple strongly augmented versions of an input into the model and encourages each output to be close to the prediction for a weakly-augmented version of the same input. To produce strong augmentations, we propose a variant of AutoAugment which learns the augmentation policy while the model is being trained. Our new algorithm, dubbed ReMixMatch, is significantly more data-efficient than prior work, requiring between $5\times$ and $16\times$ less data to reach the same accuracy. For example, on CIFAR-10 with 250 labeled examples we reach $93.73\%$ accuracy (compared to MixMatch's accuracy of $93.58\%$ with $4{,}000$ examples) and a median accuracy of $84.92\%$ with just four labels per class. We make our code and data open-source at https://github.com/google-research/remixmatch.

26 March 2021 in arXiv
Verified
Preprint
Open access
PDF file
KSKihyuk Sohn
Kihyuk Sohn
CLChun-Liang Li
Chun-Liang Li
JYJinsung Yoon
Jinsung Yoon
MJMinho Jin
Minho Jin
TPTomas Pfister
Tomas Pfister
We present a two-stage framework for deep one-class classification. We first learn self-supervised representations from one-class data, and then build one-class classifiers on learned representations. The framework not only allows to learn better representations, but also permits building one-class classifiers that are faithful to the target task. We argue that classifiers inspired by the statistical perspective in generative or discriminative models are more effective than existing approaches, such as a normality score from a surrogate classifier. We thoroughly evaluate different self-supervised representation learning algorithms under the proposed framework for one-class classification. Moreover, we present a novel distribution-augmented contrastive learning that extends training distributions via data augmentation to obstruct the uniformity of contrastive representations. In experiments, we demonstrate state-of-the-art performance on visual domain one-class classification benchmarks, including novelty and anomaly detection. Finally, we present visual explanations, confirming that the decision-making process of deep one-class classifiers is intuitive to humans. The code is available at https://github.com/google-research/deep_representation_one_class.

24 November 2020 in arXiv
Verified
Preprint
Open access
PDF file
JLJihyeon Lee
Jihyeon Lee
JXJoseph Z. Xu
Joseph Z. Xu
KSKihyuk Sohn
Kihyuk Sohn
WLWenhan Lu
Wenhan Lu
DBDavid Berthelot
David Berthelot
BKBernhard Kowatsch
Bernhard Kowatsch
To respond to disasters such as earthquakes, wildfires, and armed conflicts, humanitarian organizations require accurate and timely data in the form of damage assessments, which indicate what buildings and population centers have been most affected. Recent research combines machine learning with remote sensing to automatically extract such information from satellite imagery, reducing manual labor and turn-around time. A major impediment to using machine learning methods in real disaster response scenarios is the difficulty of obtaining a sufficient amount of labeled data to train a model for an unfolding disaster. This paper shows a novel application of semi-supervised learning (SSL) to train models for damage assessment with a minimal amount of labeled data and large amount of unlabeled data. We compare the performance of state-of-the-art SSL methods, including MixMatch and FixMatch, to a supervised baseline for the 2010 Haiti earthquake, 2017 Santa Rosa wildfire, and 2016 armed conflict in Syria. We show how models trained with SSL methods can reach fully supervised performance despite using only a fraction of labeled data and identify areas for further improvements.

1 December 2022 in arXiv
Verified
Preprint
Open access
PDF file
JYJinsung Yoon
Jinsung Yoon
KSKihyuk Sohn
Kihyuk Sohn
CLChun-Liang Li
Chun-Liang Li
SASercan O. Arik
Sercan O. Arik
TPTomas Pfister
Tomas Pfister
Semi-supervised anomaly detection is a common problem, as often the datasets containing anomalies are partially labeled. We propose a canonical framework: Semi-supervised Pseudo-labeler Anomaly Detection with Ensembling (SPADE) that isn't limited by the assumption that labeled and unlabeled data come from the same distribution. Indeed, the assumption is often violated in many applications - for example, the labeled data may contain only anomalies unlike unlabeled data, or unlabeled data may contain different types of anomalies, or labeled data may contain only 'easy-to-label' samples. SPADE utilizes an ensemble of one class classifiers as the pseudo-labeler to improve the robustness of pseudo-labeling with distribution mismatch. Partial matching is proposed to automatically select the critical hyper-parameters for pseudo-labeling without validation data, which is crucial with limited labeled data. SPADE shows state-of-the-art semi-supervised anomaly detection performance across a wide range of scenarios with distribution mismatch in both tabular and image domains. In some common real-world settings such as model facing new types of unlabeled anomalies, SPADE outperforms the state-of-the-art alternatives by 5% AUC in average.