Jin Yang

Mr. Jin Yang

orcid

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
5 July 2022 in The Journal of Supercomputing
Verified
Journal Article
Closed access
PDF file
LBLiang Bao
Liang Bao
JYJin Yang
Jin Yang
ZZZhengtong Zhang
Zhengtong Zhang
WLWenjing Liu
Wenjing Liu
JCJunhao Chen
Junhao Chen
Chase WuChase Wu
Chase Wu

24 August 2023 in IEEE Transactions on Cloud Computing
Verified
Journal Article
Closed access
JYJin Yang
Jin Yang
LBLiang Bao
Liang Bao
WLWenjing Liu
Wenjing Liu
RYRong Yang
Rong Yang
Chase WuChase Wu
Chase Wu
Deep learning (DL) has become a dominating type of workloads on AI computing platforms. The performance of such platforms highly depends on how distributed DL jobs are scheduled. Reinforcement learning (RL)-based schedulers have been extensively studied and are capable of modeling interferences between concurrent jobs competing for resources. However, existing RL-based schedulers must learn from large number of samples and adapt to workload changes in real systems, which is a huge cost for production clusters. This paper proposes an intelligent, autonomous scheduler that employs sample-efficient RL for real-world resource scheduling on complex DL clusters. Specifically, we design a closed-loop meta-RL-based worker placement algorithm for DL training jobs. Instead of random exploration, we encourage the scheduler to explore combinatorial subspaces, where the performance model might be inaccurate, to improve the sampling efficiency of the scheduler agent. Extensive experimental results demonstrate that our algorithm outperforms other baselines in terms of average job completion time with 12.29% to 16.24% improvements. Further experiments with workload variations yield 15.76% to 22.13% improvements.

22 April 2022 in Security and Communication Networks
Verified
Journal Article
Open access
PDF file
KYKan Yi
Kan Yi
JYJin Yang
Jin Yang
SWShuangling Wang
Shuangling Wang
ZZZhengtong Zhang
Zhengtong Zhang
XRXiao Ren
Xiao Ren
Service composition is a mainstream paradigm for rapidly constructing large-scale distributed applications. QoS-aware service composition, i.e., selection of the optimal execution plan that maximizes the composition’s end-to-end QoS properties, is an active area of research and development endeavors in service composition. In this paper, we propose PPDRL, a pretraining-and-policy-based deep reinforcement learning approach, to solve the QoS-aware service composition problem. Its significant feature is to incorporate a maximum likelihood estimate and a policy scoring mechanism into a deep reinforcement learning framework. As a result, our approach can balance the exploitation and exploration efforts adaptively and can search for the solution space in a robust and efficient manner. We have executed our approach to solve 6 randomly generated QoS-aware service composition problems with different sizes and structures based on QWS data set including 2,507 real Web services classified into 233 categories. The results indicate that our approach can find near-optimal solutions within moderate numbers of iteration and has performance superiority in comparison with five state-of-the-art algorithms.

12 April 2022 in Security and Communication Networks
Verified
Journal Article
Open access
PDF file
YLYiren Li
Yiren Li
TLTieke Li
Tieke Li
PSPei Shen
Pei Shen
LHLiang Hao
Liang Hao
JYJin Yang
Jin Yang
LBLiang Bao
Liang Bao
Big data analytics has become increasingly vital in many modern enterprise applications such as user profiling and business process optimization. Today’s big data processing systems, such as Hadoop MapReduce, Spark, and Hive, treat big data applications as a batch of jobs for scheduling. Existing schedulers in production systems often maintain fair allocation without considering application performance and resource utilization simultaneously. It is challenging to perform job scheduling in big data systems to achieve both low turnaround time and high resource utilization due to the high complexity in data processing logics and the dynamic variation in workloads. In this article, we propose a performance-aware scheduler, referred to as PAS, which dynamically schedules big data jobs in Hadoop YARN and Spark and autonomously adjusts scheduling policies to improve application performance and resource utilization. Specifically, PAS schedules multiple concurrent jobs using different policies based on the predicted job completion time and employs a greedy approach and a one-step lookahead strategy to opportunistically maximize the average job performance while still maintaining a satisfactory level of resource utilization. We implement PAS in Hadoop YARN and evaluate its performance with HiBench, a well-known big data processing benchmark. Experimental results show that PAS reduces the average turnaround time by 25% and the makespan by 15% in comparison with four state-of-the-art schedulers.

14 April 2022 in Mathematical Problems in Engineering
Verified
Journal Article
Open access
PDF file
KYKan Yi
Kan Yi
JYJin Yang
Jin Yang
SWShuangling Wang
Shuangling Wang
ZZZhengtong Zhang
Zhengtong Zhang
JZJing Zhang
Jing Zhang
XRXiao Ren
Xiao Ren
Enhanced index tracking (EIT) is an active research area in portfolio management that focuses on adding reliable value relative to the index on the basis of mimicking the behavior of the benchmark index. To solve the EIT problem, many approaches have been proposed. However, it still remains a critical challenge to efficiently generate a portfolio with good quality. In this study, we propose a learning-based approach named IntelliPortfolio for the EIT problem. IntelliPortfolio uses PCA and clustering to select stock and estimates the investment weight for each constituent stock using a long short-term memory (LSTM) network. Two advantages of the proposed algorithm are as follows. (1) It considers both the fundamentals and the price information for stocks and can balance the trade-off between the performance and the diversity of the selected stocks. (2) It uses a LSTM model to estimate investment weights, which is more capable to handle long sequences of input and is more robust to predict the future trend of stock market. Experimental results on the five real-world datasets of the international stock market illustrate the significant performance superiority of the proposed approach in comparison with five state-of-the-art algorithms.