Yiheng Lin

Yiheng Lin

Auto-generated profile, based on publicly available information

Yiheng Lin

California Institute of Technology, Pasadena, CA, USA

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
4 August 2024
Verified
Conference Paper
Open access
TCTianyu Chen
Tianyu Chen
YLYiheng Lin
Yiheng Lin
NCNicolas Christianson
Nicolas Christianson
ZAZahaib Akhtar
Zahaib Akhtar
SDSharath Dharmaji
Sharath Dharmaji
RSRamesh K. Sitaraman
Ramesh K. Sitaraman
The primary objective of adaptive bitrate (ABR) streaming is to enhance users' quality of experience (QoE) by dynamically adjusting the video bitrate in response to changing network conditions. However, users often find frequent bitrate switching frustrating due to the resulting inconsistency in visual quality over time, especially during live streaming when buffer lengths are short. In this paper, we propose a practical smoothness optimized dynamic adaptive (SODA) controller that specifically addresses this problem while remaining deployable. SODA is backed by theoretical guarantees and has shown superior performance in empirical evaluations. Specifically, our numerical simulations show a 9.55% to 27.8% QoE improvement and our prototype evaluation shows a 30.4% QoE improvement compared to the state-of-the-art baselines. In order to be widely deployable, SODA performs bitrate horizon planning in polynomial time compared to brute force approaches that suffer from exponential complexity. To demonstrate its real-world practicality, we deployed SODA on a wide range of devices within the production network of Amazon Prime Video. Production experiments show that SODA reduced bitrate switching by up to 88.8% and increased average stream viewing duration by up to 5.91% compared to a fine-tuned production baseline.

27 February 2023 in Proceedings of the ACM on Measurement and Analysis of Computing Systems
Verified
Journal Article
Open access
YZYizhou Zhang
Yizhou Zhang
GQGuannan Qu
Guannan Qu
PXPan Xu
Pan Xu
YLYiheng Lin
Yiheng Lin
ZCZaiwei Chen
Zaiwei Chen
AWAdam Wierman
Adam Wierman
We study a multi-agent reinforcement learning (MARL) problem where the agents interact over a given network. The goal of the agents is to cooperatively maximize the average of their entropy-regularized long-term rewards. To overcome the curse of dimensionality and to reduce communication, we propose a Localized Policy Iteration (LPI) algorithm that provably learns a near-globally-optimal policy using only local information. In particular, we show that, despite restricting each agent's attention to only its κ-hop neighborhood, the agents are able to learn a policy with an optimality gap that decays polynomially in κ. In addition, we show the finite-sample convergence of LPI to the global optimal policy, which explicitly captures the trade-off between optimality and computational complexity in choosing κ. Numerical simulations demonstrate the effectiveness of LPI.

1 February 2023 in IEEE Open Journal of Control Systems
Verified
Journal Article
Open access
TLTongxin Li
Tongxin Li
RYRuixiao Yang
Ruixiao Yang
GQGuannan Qu
Guannan Qu
YLYiheng Lin
Yiheng Lin
AWAdam Wierman
Adam Wierman
SLSteven H. Low
Steven H. Low
Machine-learned black-box policies are ubiquitous for nonlinear control problems. Meanwhile, crude model information is often available for these problems from, e.g., linear approximations of nonlinear dynamics. We study the problem of certifying a black-box control policy with stability using model-based advice for nonlinear control on a single trajectory. We first show a general negative result that a naive convex combination of a black-box policy and a linear model-based policy can lead to instability, even if the two policies are both stabilizing. We then propose an adaptive $\lambda$-confident policy , with a coefficient $\lambda$ indicating the confidence in a black-box policy, and prove its stability. With bounded nonlinearity, in addition, we show that the adaptive $\lambda$ -confident policy achieves a bounded competitive ratio when a black-box policy is near-optimal. Finally, we propose an online learning approach to implement the adaptive $\lambda$ -confident policy and verify its efficacy in case studies about the Cart-Pole problem and a real-world electric vehicle (EV) charging problem with covariate shift due to COVID-19.

19 June 2023
Verified
Conference Paper
Open access
YZYizhou Zhang
Yizhou Zhang
GQGuannan Qu
Guannan Qu
PXPan Xu
Pan Xu
YLYiheng Lin
Yiheng Lin
ZCZaiwei Chen
Zaiwei Chen
AWAdam Wierman
Adam Wierman
We study a multi-agent reinforcement learning (MARL) problem where the agents interact over a given network. The goal of the agents is to cooperatively maximize the average of their entropy-regularized long-term rewards. To overcome the curse of dimensionality and to reduce communication, we propose a Localized Policy Iteration (LPI) algorithm that provably learns a near-globally-optimal policy using only local information. In particular, we show that, despite restricting each agent's attention to only its κ-hop neighborhood, the agents are able to learn a policy with an optimality gap that decays polynomially in κ. In addition, we show the finite-sample convergence of LPI to the global optimal policy, which explicitly captures the trade-off between optimality and computational complexity in choosing κ. Numerical simulations demonstrate the effectiveness of LPI. This extended abstract is an abridged version of [12].

26 June 2023 in ACM SIGMETRICS Performance Evaluation Review
Verified
Journal Article
Open access
YZYizhou Zhang
Yizhou Zhang
GQGuannan Qu
Guannan Qu
PXPan Xu
Pan Xu
YLYiheng Lin
Yiheng Lin
ZCZaiwei Chen
Zaiwei Chen
AWAdam Wierman
Adam Wierman
We study a multi-agent reinforcement learning (MARL) problem where the agents interact over a given network. The goal of the agents is to cooperatively maximize the average of their entropy-regularized long-term rewards. To overcome the curse of dimensionality and to reduce communication, we propose a Localized Policy Iteration (LPI) algorithm that provably learns a near-globally-optimal policy using only local information. In particular, we show that, despite restricting each agent's attention to only its κ-hop neighborhood, the agents are able to learn a policy with an optimality gap that decays polynomially in κ. In addition, we show the finite-sample convergence of LPI to the global optimal policy, which explicitly captures the trade-off between optimality and computational complexity in choosing κ. Numerical simulations demonstrate the effectiveness of LPI. This extended abstract is an abridged version of [12].

3 June 2022 in arXiv
Verified
Preprint
Open access
PDF file
TLTongxin Li
Tongxin Li
RYRuixiao Yang
Ruixiao Yang
GQGuannan Qu
Guannan Qu
YLYiheng Lin
Yiheng Lin
SLSteven Low
Steven Low
AWAdam Wierman
Adam Wierman
Machine-learned black-box policies are ubiquitous for nonlinear control problems. Meanwhile, crude model information is often available for these problems from, e.g., linear approximations of nonlinear dynamics. We study the problem of equipping a black-box control policy with model-based advice for nonlinear control on a single trajectory. We first show a general negative result that a naive convex combination of a black-box policy and a linear model-based policy can lead to instability, even if the two policies are both stabilizing. We then propose an adaptive $\lambda$-confident policy, with a coefficient $\lambda$ indicating the confidence in a black-box policy, and prove its stability. With bounded nonlinearity, in addition, we show that the adaptive $\lambda$-confident policy achieves a bounded competitive ratio when a black-box policy is near-optimal. Finally, we propose an online learning approach to implement the adaptive $\lambda$-confident policy and verify its efficacy in case studies about the CartPole problem and a real-world electric vehicle (EV) charging problem with data bias due to COVID-19.