Predicting the future of complex signals The information bottleneck applied to the generalized Langevin equation

Abstract

Biological systems live in dynamic environments and thus need to adapt to changes in that environment. However, adapting costs time, so the system needs to predict its future environment based on past environments. Therefore, the system needs to store some information about this history from which it is possible to make this prediction. What information is optimal to extract from the signal history depends on the covariance function of the signal. The information bottleneck provides a framework to determine what information is relevant for this prediction. We look at a system that predicts the value of a signal generated by the generalized Langevin equation at one specific future time. The generalized Langevin equation provides a non-Markovian signal with a non-trivial memory function. We analyze two specific types of memory functions; an exponential decay, and a combination between an exponential decay and a Dirac delta function. These memory functions can generate a wide variety of signals. However, we find that the optimal response for each memory function does not vary strongly with parameter changes. The information bottleneck method requires discretization of the input signal. This discretization induces information loss, which masks what information is necessary to perform the prediction. A vector autoregressive model is constructed, which generates a discrete signal. This eliminates the information loss by the information bottleneck. As such, we find the information required to predict the future signal value for both signal types. For the exponential decay memory function, the future is predicted optimally from the most recent three signal values. The prediction seems based on determining the signal’s instantaneous position, velocity and acceleration. For the combined exponential decay and Dirac delta function memory function, the future cannot be predicted optimally from a limited set of past signal values. If the exponential decay is the more important memory function, the response mirrors the previous case but is enhanced by a strongly decaying exponential tail. This tail seems to elucidate the effect of the instantaneous Dirac delta function. On the other hand, if the Dirac delta function is the more important memory function, the response mirrors the result of the information bottleneck for the stochastic damped harmonic oscillator (Lotte Slim, Masterthesis UvA, 2020). However, it is enhanced by a slowly decaying exponential tail. The tail seems to elucidate the effect of the history-dependent exponential decay memory function.