Herbert K. H. Lee

Prof. Herbert K. H. Lee

orcid

Research Focus & Metrics

Academic Career Length

20Years (2007-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
2 January 2016 in Technometrics
Verified
Journal Article
Closed access
RGRobert Gramacy
Robert Gramacy
GGGenetha A. Gray
Genetha A. Gray
SDSébastien Le Digabel
Sébastien Le Digabel
HLHerbert K. H. Lee
Herbert K. H. Lee
PRPritam Ranjan
Pritam Ranjan
SWStefan M. Wild
Stefan M. Wild
Constrained blackbox optimization is a difficult problem, with most approaches coming from the mathematical programming literature. The statistical literature is sparse, especially in addressing problems with nontrivial constraints. This situation is unfortunate because statistical methods have many attractive properties: global scope, handling noisy objectives, sensitivity analysis, and so forth. To narrow that gap, we propose a combination of response surface modeling, expected improvement, and the augmented Lagrangian numerical optimization framework. This hybrid approach allows the statistical model to think globally and the augmented Lagrangian to act locally. We focus on problems where the constraints are the primary bottleneck, requiring expensive simulation to evaluate and substantial modeling effort to map out. In that context, our hybridization presents a simple yet effective solution that allows existing objective-oriented statistical approaches, like those based on Gaussian process surrogates and expected improvement heuristics, to be applied to the constrained setting with minor modification. This work is motivated by a challenging, real-data benchmark problem from hydrology where, even with a simple linear objective function, learning a nontrivial valid region complicates the search for a global minimum. Supplementary materials for this article are available online.

1 June 2007 in Journal of Classification
Verified
Journal Article
Closed access
PDF file
HLHerbert K. H. Lee
Herbert K. H. Lee

10 January 2024 in Stat
Verified
Journal Article
Closed access
TPTony Pourmohamad
Tony Pourmohamad
HLHerbert K. H. Lee
Herbert K. H. Lee

12 October 2025 in Mathematics
Verified
Journal Article
Open access
PDF file
NGNicholas Grunloh
Nicholas Grunloh
HLHerbert K. H. Lee
Herbert K. H. Lee
Bayesian optimization routines may have theoretical convergence results, but determining whether a run has converged in practice can be a subjective task. This paper provides a framework inspired by statistical process control for monitoring an optimization run for convergence. The maximum Expected Improvement (EI) tends to decrease during an optimization run, but decreasing EI is not sufficient for convergence. We consider both a decrease in EI as well as local stability of the variance in order to assess for convergence. The EI process is made more numerically stable through an expected log-normal approximation. An Exponentially Weighted Moving Average control chart is adapted for automated convergence analysis, which allows assessment of stability of both the EI and its variance. The success of the methodology is demonstrated on several examples.

8 December 2023
Verified
Book Chapter
Open access
PDF file
RMRui Meng
Rui Meng
HLHerbert K. H. Lee
Herbert K. H. Lee
KBKristofer Bouchard
Kristofer Bouchard
This paper presents an efficient variational inference framework for a family of structured Gaussian process regression network (SGPRN) models. We incorporate auxiliary inducing variables in latent functions and jointly treat both the distributions of the inducing variables and hyper-parameters as variational parameters. Then we take advantage of the collapsed representation of the model and propose structured variational distributions, which enables the decomposability of a tractable variational lower bound and leads to stochastic optimization. Our inference approach is able to model data in which outputs do not share a common input set, and with a computational complexity independent of the size of the inputs and outputs to easily handle datasets with missing values. Finally, we illustrate our approach on both synthetic and real data.

4 March 2026 in Journal of Classification
Verified
Journal Article
Open access
PDF file
RMRui Meng
Rui Meng
HLHerbert K. H. Lee
Herbert K. H. Lee
Latent Gaussian processes are widely used in latent variable models because they offer a flexible nonlinear mapping from a low-dimensional latent space to a high-dimensional space via Gaussian processes. However, these models are computationally intensive and do not scale well. In this paper, we propose a temporal categorical model that utilizes latent Gaussian processes, inducing-input approximation, and a regularization framework to model multivariate categorical processes, with and without priors on hyperparameters. We analyze the underlying properties and introduce two variational inference approaches: one based on a Monte Carlo method and the other on the delta method. We also found that latent dynamics tend to collapse into a constant zero, which hinders the reflection of dynamic information. To address this, we propose two strategies to regularize the latent dynamics for better alignment with observation dynamics: introducing a regularization term based on inducing variables and incorporating a scale prior for latent Gaussian processes. Additionally, we propose an efficient and effective stochastic variational inference technique. Finally, we demonstrate our model and inference methods using both synthetic data and real financial data.