Konstantinos G. Papakonstantinou

Dr. Konstantinos G. Papakonstantinou

orcid

Research Focus & Metrics

Academic Career Length

19Years (2008-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
31 May 2024 in Machine Learning
Verified
Journal Article
Open access
PDF file
GAGiacomo Arcieri
Giacomo Arcieri
CHCyprien Hoelzl
Cyprien Hoelzl
OSOliver Schwery
Oliver Schwery
DSDaniel Straub
Daniel Straub
KPKonstantinos G. Papakonstantinou
Konstantinos G. Papakonstantinou
Eleni ChatziEleni Chatzi
Eleni Chatzi
Partially Observable Markov Decision Processes (POMDPs) can model complex sequential decision-making problems under stochastic and uncertain environments. A main reason hindering their broad adoption in real-world applications is the unavailability of a suitable POMDP model or a simulator thereof. Available solution algorithms, such as Reinforcement Learning (RL), typically benefit from the knowledge of the transition dynamics and the observation generating process, which are often unknown and non-trivial to infer. In this work, we propose a combined framework for inference and robust solution of POMDPs via deep RL. First, all transition and observation model parameters are jointly inferred via Markov Chain Monte Carlo sampling of a hidden Markov model, which is conditioned on actions, in order to recover full posterior distributions from the available data. The POMDP with uncertain parameters is then solved via deep RL techniques with the parameter distributions incorporated into the solution via domain randomization, in order to develop solutions that are robust to model uncertainty. As a further contribution, we compare the use of Transformers and long short-term memory networks, which constitute model-free RL solutions and work directly on the observation space, with an approach termed the belief-input method, which works on the belief space by exploiting the learned POMDP model for belief inference. We apply these methods to the real-world problem of optimal maintenance planning for railway assets and compare the results with the current real-life policy. We show that the RL policy learned by the belief-input method is able to outperform the real-life policy by yielding significantly reduced life-cycle costs.

16 February 2018 in Structure and Infrastructure Engineering
Verified
Journal Article
Closed access
KPKonstantinos G. Papakonstantinou
Konstantinos G. Papakonstantinou
CACharalampos P. Andriotis
Charalampos P. Andriotis
MSMasanobu Shinozuka
Masanobu Shinozuka
Scheduling of inspection and maintenance policies during the life-cycle of operating infrastructure necessitates optimization of long-term objectives in stochastic environments. Modern answers to the problem should focus on quantitative decision-making techniques, taking advantage of informative but uncertain data that become available in time. As such, the problem is efficiently addressed within the framework of stochastic dynamic programming by means of Partially Observable Markov Decision Processes (POMDPs) and Mixed Observability Markov Decision Processes (MOMDPs). Although these methodologies can provide very sophisticated solutions with optimality guarantees, important computational challenges often emerge, mainly due to the continuity of the multidimensional belief space on the probability simplex. In response, recent value iteration algorithms based on point-based approaches have been suggested, focusing on reachable belief points that can support an accurate value function. In this work, several POMDP and MOMDP point-based algorithms, with various characteristics regarding the exploration of the belief space and the value function update procedures, are rigorously analyzed. The algorithms are compared and evaluated in terms of accuracy and performance in stationary and nonstationary problems of structural inspection and maintenance for life-cycle cost minimization. Results are thoroughly discussed and several insights along with practical suggestions for similar problems are provided.

27 January 2025 in Nature Communications
Verified
Journal Article
Open access
PDF file
ABAshmita Bhattacharya
Ashmita Bhattacharya
KPKonstantinos G. Papakonstantinou
Konstantinos G. Papakonstantinou
GWGordon Warn
Gordon Warn
LMLauren McPhillips
Lauren McPhillips
MBMelissa M. Bilec
Melissa M. Bilec
DCDigant Chavda
Digant Chavda
Climate change-related risk mitigation is typically addressed using cost-benefit analysis that evaluates mitigation strategies against a wide range of simulated scenarios and identifies a static policy to be implemented, without considering future observations. Due to the substantial uncertainties inherent in climate projections, this identified policy will likely be sub-optimal with respect to the actual climate trajectory that evolves in time. In this work, we thus formulate climate risk management as a dynamic decision-making problem based on Markov Decision Processes (MDPs) and Partially Observable MDPs (POMDPs), taking real-time data into account for evaluating the evolving conditions and related model uncertainties, in order to select the best possible life-cycle actions in time, with global optimality guarantees for the formulated optimization problem. The framework is developed for coastal adaptation applications, considering a wide variety of possible action types, including various forms of nature-based infrastructure. Related environmental impacts of carbon emissions and uptake are also incorporated, and social cost of carbon implications are discussed, together with several future directions and supported features.

1 March 2008
Verified
Conference Paper
Closed access
KPKonstantinos G. Papakonstantinou
Konstantinos G. Papakonstantinou
DSDirk Slock
Dirk Slock
Non-Line-of-Sight (NLOS) and multipath propagation conditions pose significant problems for most Mobile Terminal (MT) positioning approaches. This is because only path parameters of LOS paths are considered. When in fact all paths are considered, much more information for positioning becomes available, though proper consideration of the NLOS character of NLOS paths is required. On the other hand, channel parameters that have been used so far for positioning purposes concern a static channel snapshot. In the case of mobility, the path Doppler shifts provide information on the mobile terminal speed. In this paper we combine this information with traditional positioning information to jointly estimate the terminal's speed and position.

1 June 2018 in ASCE-ASME Journal of Risk and Uncertainty in Engineering Systems, Part A: Civil Engineering
Verified
Journal Article
Closed access
CVChristos Vlachos
Christos Vlachos
KPKonstantinos G. Papakonstantinou
Konstantinos G. Papakonstantinou
GDGeorge Deodatis
George Deodatis
This paper presents a novel stochastic method for simulation of ground motions. The input is a user-specified earthquake scenario description, and the output consists of fully nonstationary acceleration time histories at a site of interest. A bimodal analytical evolutionary Kanai–Tajimi (K–T) model lies at the core of the predictive stochastic model. The K–T model parameters are linked through mixed-effects regression models to three commonly used ground motion physical predictors, moment magnitude Mw , closest distance Rrup , and average soil shear-wave velocity VS30 at the site of interest. An extensive Californian subset of the next-generation attenuation NGA-West2 database is used to develop and calibrate the regression models. The random effect terms in the developed regression models effectively describe the correlation among ground motions of the same earthquake event, while also accounting for the location dependent effects at each site. The simulation of sample ground motion realizations based on each specified earthquake scenario is facilitated by the spectral representation method (SRM). In order to evaluate the performance, assess the versatility and validate the proposed predictive model, the simulation-based attenuation of important scalar ground motion engineering metrics is studied and compared with results from well-established ground motion prediction equation (GMPE) models. The statistics of elastic response spectra of simulated time histories are also compared with the associated predictions of the NGA-West2 GMPE models based on a variety of earthquake scenarios. Nonlinear response-history analyses for representative single-degree-of-freedom and multiple-degree-of-freedom structural systems compare the seismically induced inelastic structural demand of the considered systems when subjected to sets of both recorded and corresponding simulated ground motions.

6 December 2025 in Neural Networks
Verified
Journal Article
Open access
GAGiacomo Arcieri
Giacomo Arcieri
KPKonstantinos G. Papakonstantinou
Konstantinos G. Papakonstantinou
DSDaniel Straub
Daniel Straub
Eleni ChatziEleni Chatzi
Eleni Chatzi
This work introduces a novel deep learning-based architecture, termed the Deep Belief Markov Model (DBMM), which provides efficient, model-formulation agnostic inference in Partially Observable Markov Decision Process (POMDP) problems. The POMDP framework allows for modeling and solving sequential decision-making problems under observation uncertainty. In complex, high-dimensional, partially observable environments, existing methods for inference based on exact computations (e.g., via Bayes' theorem) or sampling algorithms do not always scale well. Furthermore, ground truth states may not be available for learning the exact transition dynamics. DBMMs extend deep Markov models into the partially observable decision-making framework and allow efficient belief inference entirely based on available observation data via variational inference methods. By leveraging the potency of neural networks, DBMMs can infer and simulate non-linear relationships in the system dynamics and naturally scale to problems with high dimensionality and discrete or continuous variables. In addition, neural network parameters can be dynamically updated efficiently based on data availability. DBMMs can thus be used to infer a belief variable, thus enabling the derivation of POMDP solutions over the belief space. We evaluate the efficacy of the proposed methodology by evaluating the capability of model-formulation agnostic inference of DBMMs in benchmark problems that include discrete and continuous variables. Finally, we demonstrate the practical utility of the inferred beliefs in a downstream decision-making task, showing that an RL agent guided by DBMMs beliefs significantly outperforms powerful model-free baselines and achieves near-optimal performance.1.