Time series forecasting plays a critical role in domains such as energy, finance, and healthcare, where accurate predictions inform decision-making under uncertainty. Although Transformer-based models have demonstrated success in sequential modeling, their adoption for time series remains limited by challenges such as noise sensitivity, long-range dependencies, and a lack of inductive bias for temporal structure. In this work, we present a unified and principled framework for benchmarking three prominent Transformer forecasting architectures-Autoformer, Informer, and Patchtst-each evaluated through three architectural variants: Minimal, Standard, and Full, representing increasing levels of complexity and modeling capacity. We conduct over 1500 controlled experiments on a suite of ten synthetic signals, spanning five patch lengths and five forecast horizons under both clean and noisy conditions. Our analysis reveals consistent patterns across model families. To advance this landscape further, we introduce the Koopman-enhanced Transformer framework, Deep Koopformer, which integrates operator-theoretic latent state modeling to improve stability and interpretability. We demonstrate its efficacy on nonlinear and chaotic dynamical systems. Our results highlight Koopman based Transformer as a promising hybrid approach for robust, interpretable, and theoretically grounded time series forecasting in noisy and complex real-world conditions.

Time series forecasting plays a vital role across scientific, industrial, and environmental domains, especially when dealing with high-dimensional and nonlinear systems. While Transformer-based models have recently achieved state-of-the-art performance in long-range forecasting, they often suffer from interpretability issues and instability in the presence of noise or dynamical uncertainty. In this work, we propose DeepKoopFormer, a principled forecasting framework that combines the representational power of Transformers with the theoretical rigor of Koopman operator theory. Our model features a modular encoder-propagator-decoder structure, where temporal dynamics are learned via a spectrally constrained, linear Koopman operator in a latent space. We impose structural guarantees-such as bounded spectral radius, Lyapunov based energy regularization, and orthogonal parameterization to ensure stability and interpretability. Comprehensive evaluations are conducted on both synthetic dynamical systems, real-world climate dataset (wind speed and surface pressure), financial time series (cryptocurrency), and electricity generation dataset using the Python package that is prepared for this purpose. Across all experiments, DeepKoopFormer consistently outperforms standard LSTM and baseline Transformer models in terms of accuracy, robustness to noise, and long-term forecasting stability. These results establish DeepKoopFormer as a flexible, interpretable, and robust framework for forecasting in high dimensional and dynamical settings.

In this paper, we consider the problem of system identification when side-information is available on the steady-state (or DC) gain of the system. We formulate a general nonparametric identification method as an infinite-dimensional constrained convex program over the reproducing kernel Hilbert space (RKHS) of stable impulse responses. The objective function of this optimization problem is the empirical loss regularized with the norm of RKHS, and the constraint is considered for enforcing the integration of the steady-state gain side-information. The proposed formulation addresses both the discrete-time and continuous-time cases. We show that this program has a unique solution obtained by solving an equivalent finite-dimensional convex optimization. This solution has a closed-form when the empirical loss and regularization functions are quadratic and exact side-information is considered. We perform extensive numerical comparisons to verify the efficiency of the proposed identification methodology.

In this paper, we discuss the problem of system identification when frequency domain side information is available on the system. Initially, we consider the case where the prior knowledge is provided as being the $\Hcal_{\infty}$-norm of the system bounded by a given scalar. This framework provides the opportunity of considering various forms of side information such as the dissipativity of the system as well as other forms of frequency domain prior knowledge. We propose a nonparametric identification method for estimating the impulse response of the system under the given side information. The estimation problem is formulated as an optimization in a reproducing kernel Hilbert space (RKHS) endowed with a stable kernel. The corresponding objective function consists of a term for minimizing the fitting error, and a regularization term defined based on the norm of the impulse response in the employed RKHS. To guarantee the desired frequency domain features defined based on the prior knowledge, suitable constraints are imposed on the estimation problem. The resulting optimization has an infinite-dimensional feasible set with an infinite number of constraints. We show that this problem is a well-defined convex program with a unique solution. We propose a heuristic that tightly approximates this unique solution. The proposed approach is equivalent to solving a finite-dimensional convex quadratically constrained quadratic program. The efficiency of the discussed method is verified by several numerical examples.

In recent years, the reproducing kernel Hilbert space (RKHS) theory has played a crucial role in linear system identification. The core of a RKHS is the associated kernel characterizing its properties. Accordingly, this work studies the class of diagonally square root integrable (DSRI) kernels. We demonstrate that various well-known stable kernels introduced in system identification belong to this category. Moreover, it is shown that any DSRI kernel is also stable and integrable. We look into certain topological features of the RKHSs associated with DSRI kernels, particularly the continuity of linear operators defined on the respective RKHSs. For the stability of a Gaussian process centered at a stable impulse response, we show that the necessary and sufficient condition is the diagonally square root integrability of the corresponding kernel. Furthermore, we elaborate on this result by providing proper interpretations.

In this paper, we address the identification problem for the systems characterized by linear time-invariant dynamics with bilinear observation models. More precisely, we consider a suitable parametric description of the system and formulate the identification problem as the estimation of the parameters defining the mathematical model of the system using the observed input-output data. To this end, we propose two probabilistic frameworks. The first framework employs the Maximum Likelihood (ML) approach, which accurately finds the optimal parameter estimates by maximizing a likelihood function. Subsequently, we develop a tractable first-order method to solve the optimization problem corresponding to the proposed ML approach. Additionally, to further improve tractability and computational efficiency of the estimation of the parameters, we introduce an alternative framework based on the Expectation--Maximization (EM) approach, which estimates the parameters using an appropriately designed cost function. We show that the EM cost function is invex, which ensures the existence and uniqueness of the optimal solution. Furthermore, we derive the closed-form solution for the optimal parameters and also prove the recursive feasibility of the EM procedure. Through extensive numerical experiments, the practical implementation of the proposed approaches is demonstrated, and their estimation efficacy is verified and compared, highlighting the effectiveness of the methods to accurately estimate the system parameters and their potential for real-world applications in scenarios involving bilinear observation structures.