In this work, we developed a nonlinear System Identification (SID) method that we called Entropic Regression. Our method adopts an information-theoretic measure for the data-driven discovery of the underlying dynamics. Our method shows robustness toward noise and outliers, and it outperforms many of the current state-of-the-art methods. Moreover, the method of Entropic Regression overcomes many of the major limitations of the current methods such as sloppy parameters, diverse scale, and SID in high-dimensional systems such as complex networks. The use of information-theoretic measures in entropic regression has unique advantages, due to the Asymptotic Equipartition Property of probability distributions, that outliers and other low-occurrence events are conveniently and intrinsically de-emphasized as not-typical, by definition. We provide a numerical comparison with the current state-of-the-art methods in sparse regression, and we apply the methods to different chaotic systems such as the Lorenz System, the Kuramoto-Sivashinsky equations, and the Double-Well Potential.
1
1 November 2021 in Chaos: An Interdisciplinary Journal of Nonlinear Science
The ultimate goal of cognitive neuroscience is to understand the mechanistic neural processes underlying the functional organization of the brain. The key to this study is understanding the structure of both the structural and functional connectivity between anatomical regions. In this paper, we use an information theoretic approach, which defines direct information flow in terms of causation entropy, to improve upon the accuracy of the recovery of the true network structure over popularly used methods for this task such as correlation and least absolute shrinkage and selection operator regression. The method outlined above is tested on synthetic data, which is produced by following previous work in which a simple dynamical model of the brain is used, simulated on top of a real network of anatomical brain regions reconstructed from diffusion tensor imaging. We demonstrate the effectiveness of the method of AlMomani et al. [Chaos 30, 013107 (2020)] when applied to data simulated on the realistic diffusion tensor imaging network, as well as on randomly generated small-world and Erdös–Rényi networks.
1 December 2023 in Chaos: An Interdisciplinary Journal of Nonlinear Science
The recent introduction of geometric partition entropy offered an alternative to differential Shannon entropy for the quantification of uncertainty as estimated from a sample drawn from a one-dimensional bounded continuous probability distribution. In addition to being a fresh perspective for the basis of continuous information theory, this new approach provided several improvements over traditional entropy estimators including its effectiveness on sparse samples and a proper incorporation of the impact from extreme outliers. However, a complimentary relationship exists between the new geometric approach and the basic form of its frequency-based predecessor that is leveraged here to define an entropy measure with no bias toward the sample size. This stable normalized measure is named the Boltzmann–Shannon interaction entropy (BSIE)) as it is defined in terms of a standard divergence between the measure-based and frequency-based distributions that can be associated with the two historical figures. This parameter-free measure can be accurately estimated in a computationally efficient manner, and we illustrate its utility as a quality metric for subsampling in the context of nonlinear polynomial regression.
Our recently developed tool, called Directed Affinity Segmentation (DAS), was originally designed for the data-driven discovery of coherent sets in fluidic systems. Here we interpret that it can also be used to indicate early warning signs of critical transitions in ice shelves as seen from remote sensing data. We apply a directed spectral clustering methodology, including an asymmetric affinity matrix and the associated directed graph Laplacian, to reprocess the ice velocity data and remote sensing satellite images of the Larsen C ice shelf. Our tool has enabled the simulated prediction of historical events from historical data and fault lines responsible for the critical transitions leading to the breakup of the Larsen C ice shelf crack, which resulted in the A-68 iceberg. Such benchmarking of methods, using data from the past to forecast events that are now also in the past, is sometimes called post-casting, analogous to forecasting into the future. Our method indicated the coming crisis months before the actual occurrence.
Network optimization strategies for the process of synchronization have generally focused on the re-wiring or re-weighting of links in order to (1) expand the range of coupling strengths that achieve synchronization, (2) expand the basin of attraction for the synchronization manifold, or (3) lower the average time to synchronization. A new optimization goal is proposed in seeking the minimum subset of the edge set of the original network that enables the same essential ability to synchronize in that the synchronization manifolds have conjugate stability. We call this type of minimal spanning subgraph an essential synchronization backbone of the original system, and we present two algorithms: one is a strategy for an exhaustive search for a true solution, while the other is a method of approximation for this combinatorial problem. The solution spaces that result from different choices of dynamical systems and coupling schemes vary with the level of a hierarchical structure present and also the number of interwoven central cycles. Applications can include the important problem in civil engineering of power grid hardening, where new link creation may be costly, and the defense of certain key links to the functional process may be prioritized.