Lillian J. Ratliff

Lillian J. Ratliff

Auto-generated profile, based on publicly available information

Lillian J. Ratliff

University of Washington

Research Focus & Metrics

Academic Career Length

11Years (2016-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
29 September 2020 in IEEE Transactions on Automatic Control
Verified
Journal Article
Closed access
LRLillian J. Ratliff
Lillian J. Ratliff
TFTanner Fiez
Tanner Fiez
We apply control theoretic and optimization techniques to adaptively design incentives for principal-agent problems in which the principal faces adverse selection in its interaction with multiple agents. In particular, the principal's objective depends on data from strategic decision makers (agents) whose decision-making process is unknown a priori . We consider both the cases where agents play best response to one another (Nash) and where they employ myopic update rules. By parametrizing the agents’ utility functions and the incentives offered, we develop an algorithm that the principal can employ to learn the agents’ decision-making processes while simultaneously designing incentives to change their response to one that is more desirable. We provide convergence results for this algorithm both in the noise-free and noisy cases and present illustrative examples.

3 July 2019 in IEEE Transactions on Automatic Control
Verified
Journal Article
Closed access
LRLillian J. Ratliff
Lillian J. Ratliff
EMEric Mazumdar
Eric Mazumdar
This work addresses the problem of inverse reinforcement learning in Markov decision processes where the decision-making agent is risk-sensitive . In particular, a risk-sensitive reinforcement learning algorithm with convergence guarantees that makes use of coherent risk metrics and models of human decision-making which have their origins in behavioral psychology and economics is presented. The risk-sensitive reinforcement learning algorithm provides the theoretical underpinning for a gradient-based inverse reinforcement learning algorithm that seeks to minimize a loss function defined on the observed behavior. It is shown that the gradient of the loss function with respect to the model parameters is well defined and computable via a contraction map argument. Evaluation of the proposed technique is performed on a Grid World example, a canonical benchmark problem.

3 November 2020 in IEEE Transactions on Intelligent Transportation Systems
Verified
Journal Article
Closed access
ELEsther Ling
Esther Ling
LZLiyuan Zheng
Liyuan Zheng
LRLillian J. Ratliff
Lillian J. Ratliff
SCSamuel Coogan
Samuel Coogan
This paper proposes Koopman operator theory and the related algorithm dynamical mode decomposition (DMD) for analysis and control of signalized traffic flow networks. DMD provides a model-free approach for representing complex oscillatory dynamics from measured data, and we study its application to several problems in signalized traffic. We first study a single signalized intersection, and we propose applying this method to infer traffic signal control parameters such as phase timing directly from traffic flow data. Next, we propose using the oscillatory modes of the Koopman operator, approximated with DMD, for early identification of unstable queue growth that has the potential to cause cascading congestion. Then we demonstrate how DMD can be coupled with knowledge of the traffic signal control status to determine traffic signal control parameters that are able to reduce queue lengths. Lastly, we demonstrate that DMD allows for determining the structure and the strength of interactions in a network of signalized intersections. All examples are demonstrated using a case study network instrumented with high resolution traffic flow sensors.

23 May 2017 in IEEE Transactions on Control Systems Technology
Verified
Journal Article
Closed access
IKIoannis C. Konstantakopoulos
Ioannis C. Konstantakopoulos
LRLillian J. Ratliff
Lillian J. Ratliff
MJMing Jin
Ming Jin
SSS. Shankar Sastry
S. Shankar Sastry
CSCostas J. Spanos
Costas J. Spanos
In many smart infrastructure applications, flexibility in achieving sustainability goals can be gained by engaging end users. However, these users often have heterogeneous preferences that are unknown to the decision maker tasked with improving operational efficiency. Modeling user interaction as a continuous game between noncooperative players, we propose a robust parametric utility learning framework that employs constrained feasible generalized least squares estimation with heteroskedastic inference. To improve forecasting performance, we extend the robust utility learning scheme by employing bootstrapping with bagging, bumping, and gradient boosting ensemble methods. Moreover, we estimate the noise covariance, which provides approximated correlations between players, which we leverage to develop a novel correlated utility learning framework. We apply the proposed methods both to a toy example arising from Bertrand-Nash competition between two firms and to data from a social game experiment designed to encourage energy efficient behavior among smart building occupants. Using occupant voting data for shared resources such as lighting, we simulate the game defined by the estimated utility functions to demonstrate the performance of the proposed methods.

11 June 2019 in IEEE Transactions on Automatic Control
Verified
Journal Article
Closed access
TWTyler Westenbroek
Tyler Westenbroek
RDRoy Dong
Roy Dong
LRLillian J. Ratliff
Lillian J. Ratliff
SSS. Shankar Sastry
S. Shankar Sastry
In recent years, data has played an increasingly important role in the economy as a good in its own right. In many settings, data aggregators cannot directly verify the quality of the data they purchase, nor the effort exerted by data sources when creating the data. Recent work has explored mechanisms to ensure that the data sources share high quality data with a single data aggregator, addressing the issue of moral hazard. Oftentimes, there is a unique, socially efficient solution.

24 December 2019 in IEEE Transactions on Automatic Control
Verified
Journal Article
Closed access
SSShreyas Sekar
Shreyas Sekar
LZLiyuan Zheng
Liyuan Zheng
LRLillian J. Ratliff
Lillian J. Ratliff
BZBaosen Zhang
Baosen Zhang
We study the equilibrium behavior in a multi-commodity selfish routing game with uncertain users where each user over- or under-estimates their congestion costs by a multiplicative factor. Surprisingly, we find that uncertainties in different directions have qualitatively distinct impacts on equilibria. Namely, contrary to the usual notion that uncertainty increases inefficiencies, network congestion decreases when users over-estimate their costs. On the other hand, under-estimation of costs leads to increased congestion.