Statistical mechanics is used to describe the observed information processing complexity of nonlinear dynamical systems. We introduce a measure of complexity distinct from and dual to the information theoretic entropies and dimensions. A technique is presented that directly reconstructs minimal equations of motion from the recursive structure of measurement sequences. Application to the period-doubling cascade demonstrates a form of superuniversality that refers only to the entropy and complexity of a data stream.

We introduce and analyze a general model of a population evolving over a network of selectively neutral genotypes. We show that the population’s limit distribution on the neutral network is solely determined by the network topology and given by the principal eigenvector of the network’s adjacency matrix. Moreover, the average number of neutral mutant neighbors per individual is given by the matrix spectral radius. These results quantify the extent to which populations evolve mutational robustness—the insensitivity of the phenotype to mutations—and thus reduce genetic load. Because the average neutrality is independent of evolutionary parameters—such as mutation rate, population size, and selective advantage—one can infer global statistics of neutral network topology by using simple population data available fromin vitroorin vivoevolution. Populations evolving on neutral networks of RNA secondary structures show excellent agreement with our theoretical predictions.

We study how the Shannon entropy of sequences produced by an information source converges to the source’s entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to stochastic and deterministic processes by using successive derivatives of the Shannon entropy growth curve. This leads, in turn, to natural measures of apparent memory stored in a source and the amounts of information that must be extracted from observations of a source in order for it to be optimally predicted and for an observer to synchronize to it. To measure the difficulty of synchronization, we define the transient information and prove that, for Markov processes, it is related to the total uncertainty experienced while synchronizing to a process. One consequence of ignoring a process’s structural properties is that the missed regularities are converted to apparent randomness. We demonstrate that this problem arises particularly for settings where one has access only to short measurement sequences. Numerically and analytically, we determine the Shannon entropy growth curve, and related quantities, for a range of stochastic and deterministic processes. We conclude by looking at the relationships between a process’s entropy convergence behavior and its underlying computational structure.