Jun Huan

Jun Huan

Auto-generated profile, based on publicly available information

Jun Huan

Amazon Web Services, Santa Clara, CA, USA

Research Focus & Metrics

Academic Career Length

9Years (2018-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
8 September 2022 in ACM Transactions on Knowledge Discovery From Data
Verified
Journal Article
Open access
HXHaoyi Xiong
Haoyi Xiong
RWRuosi Wan
Ruosi Wan
Jian ZhaoJian Zhao
Jian Zhao
ZCZeyu Chen
Zeyu Chen
XLXingjian Li
Xingjian Li
JHJun Huan
Jun Huan
Regularization that incorporates the linear combination of empirical loss and explicit regularization terms as the loss function has been frequently used for many machine learning tasks. The explicit regularization term is designed in different types, depending on its applications. While regularized learning often boost the performance with higher accuracy and faster convergence, the regularization would sometimes hurt the empirical loss minimization and lead to poor performance. To deal with such issues in this work, we propose a novel strategy, namely Gr adients O rthogonal D ecomposition ( GrOD ), that improves the training procedure of regularized deep learning. Instead of linearly combining gradients of the two terms, GrOD re-estimates a new direction for iteration that does not hurt the empirical loss minimization while preserving the regularization affects, through orthogonal decomposition. We have performed extensive experiments to use GrOD improving the commonly used algorithms of transfer learning [ 2 ], knowledge distillation [ 3 ], and adversarial learning [ 4 ]. The experiment results based on large datasets, including Caltech 256 [ 5 ], MIT indoor 67 [ 6 ], CIFAR-10 [ 7 ], and ImageNet [ 8 ], show significant improvement made by GrOD for all three algorithms in all cases.

20 November 2019 in IEEE Transactions on Knowledge and Data Engineering
Verified
Journal Article
Closed access
XCXiaoyang Chen
Xiaoyang Chen
HHHongwei Huo
Hongwei Huo
JHJun Huan
Jun Huan
JVJeffrey Scott Vitter
Jeffrey Scott Vitter
WZWeiguo Zheng
Weiguo Zheng
LZLei Zou
Lei Zou
Graph similarity search has received considerable attention in many applications, such as bioinformatics, pattern recognition and social networks. Existing methods have limited scalability because of the huge amount of memory they consume when handling very large graph databases with tens of millions of graphs. In this paper, we present a succinct index that incorporates succinct data structures and hybrid encoding to achieve improved query time performance with minimal space usage. Specifically, the space usage of our index requires only 5%-15% of the previous state-of-the-art indexing size while at the same time achieving several times acceleration in query time on the tested data. We also improve the query performance by augmenting the global filter with range searching, which allows us to search in a reduced region. Besides, we propose two effective lower bounds together with a boosting technique to obtain the possible smallest candidate set. Extensive experiments demonstrate that our proposed approach is superior both in space and filtering to the state-of-the-art approaches. To the best of our knowledge, our index is the first in-memory index that successfully scales to cope with the large dataset of 25 million chemical graphs from the PubChem dataset. The source code is available online.

1 July 2020 in Neurocomputing
Verified
Journal Article
Closed access
Zhigang SunZhigang Sun
Zhigang Sun
HHHongwei Huo
Hongwei Huo
JHJun Huan
Jun Huan
JVJeffrey Scott Vitter
Jeffrey Scott Vitter

4 August 2023
Verified
Conference Paper
Open access
AMAashiq Muhamed
Aashiq Muhamed
CBChristian Bock
Christian Bock
RSRahul Solanki
Rahul Solanki
YPYoungsuk Park
Youngsuk Park
YWYida Wang
Yida Wang
JHJun Huan
Jun Huan
Foundation models such as ChatGPT and GPT-4 have garnered significant interest from both academia and industry due to their emergent capabilities, such as few-shot prompting, multi-step reasoning, instruction following, and model calibration. Such capabilities were previously only attainable with specially designed models, such as those using knowledge graphs, but can now be achieved on a much larger scale with foundation models. As the capabilities of foundation models have increased, so too have their sizes at a rate much faster than Moore's law. For example, the BERT large model was initially released as a 334M model in 2018, and by 2023, the largest GPT-4 models are estimated to range between 200-300B, representing an increase of three orders of magnitude in just five years. The training of foundation models requires massive computing power. For instance, training a BERT model on a single state-of-the-art GPU machine with multi-A100 chips can take several days, while training GPT-3 models on a large multi-instance GPU cluster can take several months to complete the estimated 3 X 1023 flops. This tutorial provides an overview of the latest progress in supporting foundation model training and inference with new AI chips. It reviews progress on the modeling side, with an emphasis on the transformer architecture, and presents the system architecture supporting training and serving foundation models. This includes programming language frameworks such as PyTorch and Tensorflow, graph compilers, 3D parallelisms, and accelerators such as the GPU H100, TPU, and Trainium. Finally, the tutorial presents our experience of training foundation models using different systems.

17 July 2019 in Proceedings of the AAAI Conference on Artificial Intelligence
Verified
Journal Article
Open access
PDF file
HKHannah Kim
Hannah Kim
DKDenys Katerenchuk
Denys Katerenchuk
DBDaniel Billet
Daniel Billet
JHJun Huan
Jun Huan
HPHaesun Park
Haesun Park
BLBoyang Li
Boyang Li
Understanding narrative content has become an increasingly popular topic. Nonetheless, research on identifying common types of narrative characters, or personae, is impeded by the lack of automatic and broad-coverage evaluation methods. We argue that computationally modeling actors provides benefits, including novel evaluation mechanisms for personae. Specifically, we propose two actor-modeling tasks, cast prediction and versatility ranking, which can capture complementary aspects of the relation between actors and the characters they portray. For an actor model, we present a technique for embedding actors, movies, character roles, genres, and descriptive keywords as Gaussian distributions and translation vectors, where the Gaussian variance corresponds to actors’ versatility. Empirical results indicate that (1) the technique considerably outperforms TransE (Bordes et al. 2013) and ablation baselines and (2) automatically identified persona topics (Bamman, O’Connor, and Smith 2013) yield statistically significant improvements in both tasks, whereas simplistic persona descriptors including age and gender perform inconsistently, validating prior research.

7 September 2018 in arXiv
Verified
Preprint
Open access
PDF file
TWTianyang Wang
Tianyang Wang
JHJun Huan
Jun Huan
Bo LiBo Li
Bo Li
Deep learning models learn to fit training data while they are highly expected to generalize well to testing data. Most works aim at finding such models by creatively designing architectures and fine-tuning parameters. To adapt to particular tasks, hand-crafted information such as image prior has also been incorporated into end-to-end learning. However, very little progress has been made on investigating how an individual training sample will influence the generalization ability of a model. In other words, to achieve high generalization accuracy, do we really need all the samples in a training dataset? In this paper, we demonstrate that deep learning models such as convolutional neural networks may not favor all training samples, and generalization accuracy can be further improved by dropping those unfavorable samples. Specifically, the influence of removing a training sample is quantifiable, and we propose a Two-Round Training approach, aiming to achieve higher generalization accuracy. We locate unfavorable samples after the first round of training, and then retrain the model from scratch with the reduced training dataset in the second round. Since our approach is essentially different from fine-tuning or further training, the computational cost should not be a concern. Our extensive experimental results indicate that, with identical settings, the proposed approach can boost performance of the well-known networks on both high-level computer vision problems such as image classification, and low-level vision problems such as image denoising.