28 June 2022 in Physical Review Accelerators and Beams
Tuning machine parameters of particle accelerators is a repetitive and time-consuming task that is challenging to automate. While many off-the-shelf optimization algorithms are available, in practice their use is limited because most methods do not account for safety-critical constraints in each iteration, such as loss signals or step-size limitations. One notable exception is safe Bayesian optimization, which is a data-driven tuning approach for global optimization with noisy feedback. We propose and evaluate a step-size limited variant of safe Bayesian optimization on two research facilities of the PSI: (a) the SwissFEL and (b) HIPA. We report promising experimental results on both machines, tuning up to 16 parameters subject to 224 constraints.
3 April 2020 in Proceedings of the AAAI Conference on Artificial Intelligence
Bayesian optimization and kernelized bandit algorithms are widely used techniques for sequential black box function optimization with applications in parameter tuning, control, robotics among many others. To be effective in high dimensional settings, previous approaches make additional assumptions, for example on low-dimensional subspaces or an additive structure. In this work, we go beyond the additivity assumption and use an orthogonal projection pursuit regression model, which strictly generalizes additive models. We present a two-stage algorithm motivated by experimental design to first decorrelate the additive components. Subsequently, the bandit optimization benefits from the statistically efficient additive model. Our method provably decorrelates the fully additive model and achieves optimal sublinear simple regret in terms of the number of function evaluations. To prove the rotation recovery, we derive novel concentration inequalities for linear regression on subspaces. In addition, we specifically address the issue of acquisition function optimization and present two domain dependent efficient algorithms. We validate the algorithm numerically on synthetic as well as real-world optimization problems.
10 November 2026 in IEEE Robotics and Automation Letters
19 January 2026 in International Journal of Wildland Fire
Background: Wildfire policy effectiveness is difficult to evaluate, because fire activity depends on multiple interacting factors. Aims: We conduct a data analysis study to determine (1) if synthetic control estimations are a suitable methodology for detecting the effect of wildfire policy interventions, and (2) if shifts in the wildfire regime in Italy can be attributed to the ‘Madia’s law’ policy intervention that in 2017 changed agency responsibility for firefighting in most regions. Methods: We use synthetic controls with and without consideration of fire weather to model fire activity in aggregated and individual regions in Italy that were subject to the policy intervention, using a control pool of European countries. Key results: Synthetic control is demonstrated as a suitable approach to model counterfactual trends in fire activity after a policy intervention. In the case of Madia’s law, models support the attribution of higher burned area and average fire size to the policy intervention in the first year after its implementation, though this effect appears to a varying extent across regions. Conclusions: Data-driven approaches offer insights into policy effectiveness, but challenges remain owing to complex interacting factors. Implications: Synthetic controls can complement expert-based assessments of wildfire policies in a range of flammable landscapes.
13 November 2023 in arXiv
Partial monitoring is an expressive framework for sequential decision-making with an abundance of applications, including graph-structured and dueling bandits, dynamic pricing and transductive feedback models. We survey and extend recent results on the linear formulation of partial monitoring that naturally generalizes the standard linear bandit setting. The main result is that a single algorithm, information-directed sampling (IDS), is (nearly) worst-case rate optimal in all finite-action games. We present a simple and unified analysis of stochastic partial monitoring, and further extend the model to the contextual and kernelized setting.
19 December 2022 in arXiv
Many real-world reinforcement learning tasks require control of complex dynamical systems that involve both costly data acquisition processes and large state spaces. In cases where the transition dynamics can be readily evaluated at specified states (e.g., via a simulator), agents can operate in what is often referred to as planning with a \emph{generative model}. We propose the AE-LSVI algorithm for best-policy identification, a novel variant of the kernelized least-squares value iteration (LSVI) algorithm that combines optimism with pessimism for active exploration (AE). AE-LSVI provably identifies a near-optimal policy \emph{uniformly} over an entire state space and achieves polynomial sample complexity guarantees that are independent of the number of states. When specialized to the recently introduced offline contextual Bayesian optimization setting, our algorithm achieves improved sample complexity bounds. Experimentally, we demonstrate that AE-LSVI outperforms other RL algorithms in a variety of environments when robustness to the initial state is required.