Optimization, particularly constrained optimization problems (COPs), is fundamental in engineering, influencing various sectors with its critical role in enhancing design efficiency, reducing experimental costs, and shortening testing cycles. This study explores the challenges inherent in COPs, with a focus on developing efficient solution methodologies under stringent constraints. Surrogate models, especially Gaussian Process Regression (GPR), are pivotal in our approach, enabling the approximation of complex systems with reduced computational demand. We evaluate the efficacy of the Efficient Global Optimization (EGO) algorithm, which synergizes GPR with the Expected Improvement (EI) function, and further extend this framework to Constrained Expected Improvement (CEI) and our novel methodology Constrained Expected Prediction Error (CEPE). We demonstrate the effectiveness of these methodologies by numerical benchmark simulations and the real-world application of optimizing a Three-Bar Truss Design. In essence, the innovative CEPE approach promises a potent balance between solution accuracy and computational prowess, offering significant potential in the broader engineering field.

Background: Multi-head self-attention is central to Transformer-based time-series forecasting, yet its head-level information-selection behavior lacks a unified information-theoretic characterization. How much information a single head selects, how inter-head redundancy should be measured, and under what conditions a head can be removed without degrading predictions remain open questions. Methods: We treat each attention head as a discrete auxiliary selection channel whose conditional distribution is the attention weight vector. This yields a closed-form information identity and an entropy-dependent upper bound on selection information: $I(X;J_t^h)\le logL-\mathbb{E}\left[H\left({\alpha }_t^h\right)\right]$. We introduce total correlation—the Kullback–Leibler divergence between the joint head distribution and the product of its marginals—as a distributionally principled redundancy measure and relate head-removal sensitivity to conditional task information under population log-loss. Importantly, the selection-information bound characterizes input-dependent positional selection induced by attention weights, rather than the task information carried by the continuous value-weighted head output. Results: Synthetic experiments confirm the entropy-regularized optimality of softmax attention, the selection-information bound, and the redundancy decomposition under controlled conditions. Time-series forecasting experiments across nine benchmark datasets reveal that the head count achieving the lowest observed mean MSE varies across datasets, and that redundancy–sensitivity relationships are dataset- and head-count-dependent, though none remains statistically significant after multiple-comparison correction. Conclusions: The framework provides a principled diagnostic tool for analyzing selection behavior, inter-head dependence, and head-removal sensitivity in multi-head self-attention. It is a diagnostic framework rather than a new forecasting architecture or a standalone pruning algorithm. Pairwise redundancy carries diagnostic signal but is not, by itself, a complete predictor of head-removal sensitivity.