Henning Sprekeler

Henning Sprekeler

Auto-generated profile, based on publicly available information

Henning Sprekeler

Institute of Software Engineering and Theoretical Computer Science, Technical University of Berlin, Berlin, Germany

Research Focus & Metrics

Academic Career Length

20Years (2007-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
16 December 2011 in Science
Verified
Journal Article
Open access
PDF file
TVT. P. Vogels
T. P. Vogels
HSH. Sprekeler
H. Sprekeler
FZF. Zenke
F. Zenke
CCC. Clopath
C. Clopath
WGW. Gerstner
W. Gerstner
Plasticity at inhibitory synapses maintains balanced excitatory and inhibitory synaptic inputs at cortical neurons.

11 April 2013 in PLOS Computational Biology
Verified
Journal Article
Open access
PDF file
NFNicolas Frémaux
Nicolas Frémaux
HSHenning Sprekeler
Henning Sprekeler
WGWulfram Gerstner
Wulfram Gerstner
Animals repeat rewarded behaviors, but the physiological basis of reward-based learning has only been partially elucidated. On one hand, experimental evidence shows that the neuromodulator dopamine carries information about rewards and affects synaptic plasticity. On the other hand, the theory of reinforcement learning provides a framework for reward-based learning. Recent models of reward-modulated spike-timing-dependent plasticity have made first steps towards bridging the gap between the two approaches, but faced two problems. First, reinforcement learning is typically formulated in a discrete framework, ill-adapted to the description of natural situations. Second, biologically plausible models of reward-modulated spike-timing-dependent plasticity require precise calculation of the reward prediction error, yet it remains to be shown how this can be computed by neurons. Here we propose a solution to these problems by extending the continuous temporal difference (TD) learning of Doya (2000) to the case of spiking neurons in an actor-critic network operating in continuous time, and with continuous state and action representations. In our model, the critic learns to predict expected future rewards in real time. Its activity, together with actual rewards, conditions the delivery of a neuromodulatory TD signal to itself and to the actor, which is responsible for action choice. In simulations, we show that such an architecture can solve a Morris water-maze-like navigation task, in a number of trials consistent with reported animal performance. We also use our model to solve the acrobot and the cartpole problems, two complex motor control tasks. Our model provides a plausible way of computing reward prediction error in the brain. Moreover, the analytically derived learning rule is consistent with experimental evidence for dopamine-modulated spike-timing-dependent plasticity. As every dog owner knows, animals repeat behaviors that earn them rewards. But what is the brain machinery that underlies this reward-based learning? Experimental research points to plasticity of the synaptic connections between neurons, with an important role played by the neuromodulator dopamine, but the exact way synaptic activity and neuromodulation interact during learning is not precisely understood. Here we propose a model explaining how reward signals might interplay with synaptic plasticity, and use the model to solve a simulated maze navigation task. Our model extends an idea from the theory of reinforcement learning: one group of neurons form an “actor,” responsible for choosing the direction of motion of the animal. Another group of neurons, the “critic,” whose role is to predict the rewards the actor will gain, uses the mismatch between actual and expected reward to teach the synapses feeding both groups. Our learning agent learns to reliably navigate its maze to find the reward. Remarkably, the synaptic learning rule that we derive from theoretical considerations is similar to previous rules based on experimental evidence.

6 October 2010 in The Journal of Neuroscience
Verified
Journal Article
Open access
PDF file
NFNicolas Frémaux
Nicolas Frémaux
HSHenning Sprekeler
Henning Sprekeler
WGWulfram Gerstner
Wulfram Gerstner
Recent experiments have shown that spike-timing-dependent plasticity is influenced by neuromodulation. We derive theoretical conditions for successful learning of reward-related behavior for a large class of learning rules where Hebbian synaptic plasticity is conditioned on a global modulatory factor signaling reward. We show that all learning rules in this class can be separated into a term that captures the covariance of neuronal firing and reward and a second term that presents the influence of unsupervised learning. The unsupervised term, which is, in general, detrimental for reward-based learning, can be suppressed if the neuromodulatory signal encodes the difference between the reward and the expected reward—but only if the expected reward is calculated for each task and stimulus separately. If several tasks are to be learned simultaneously, the nervous system needs an internal critic that is able to predict the expected reward for arbitrary stimuli. We show that, with a critic, reward-modulated spike-timing-dependent plasticity is capable of learning motor trajectories with a temporal resolution of tens of milliseconds. The relation to temporal difference learning, the relevance of block-based learning paradigms, and the limitations of learning with a critic are discussed.

31 August 2007 in PLOS Computational Biology
Verified
Journal Article
Open access
PDF file
MFMathias Franzius
Mathias Franzius
HSHenning Sprekeler
Henning Sprekeler
LWLaurenz Wiskott
Laurenz Wiskott
We present a model for the self-organized formation of place cells, head-direction cells, and spatial-view cells in the hippocampal formation based on unsupervised learning on quasi-natural visual stimuli. The model comprises a hierarchy of Slow Feature Analysis (SFA) nodes, which were recently shown to reproduce many properties of complex cells in the early visual system [1]. The system extracts a distributed grid-like representation of position and orientation, which is transcoded into a localized place-field, head-direction, or view representation, by sparse coding. The type of cells that develops depends solely on the relevant input statistics, i.e., the movement pattern of the simulated animal. The numerical simulations are complemented by a mathematical analysis that allows us to accurately predict the output of the top SFA layer. Rats excel at navigating through complex environments. In order to find their way, they need to answer two basic questions. Where am I? In which direction am I heading? As the brain has no direct access to information about its position in space, it has to rely on sensory signals—from eyes and ears for example—to answer these questions. Information about its position and orientation is typically present in the information it gathers from its senses, but unfortunately it is encoded in a way that is not obvious to decode. Three major types of cells in the brain whose firing directly reflects spatial information are place, head-direction, and view cells. Place cells, for example, fire when the animal is at a particular location independent of the direction the animal is looking in. In this study, we present a self-organizational model that develops these three representation types by learning on naturalistic videos mimicking the visual input of a rat. Although the model works on complex visual stimuli, a rigorous mathematical description of the system is given as well.

22 June 2018 in Proceedings of the National Academy of Sciences
Verified
Journal Article
Open access
PDF file
RNRichard Naud
Richard Naud
HSHenning Sprekeler
Henning Sprekeler
Significance: Understanding the neural code is to attribute proper meaning to temporal sequences of action potentials. We report a simple neural code based on distinguishing single spikes from spikes in close succession, commonly called “bursts.” By separating these two types of responses, we show that ensembles of neurons can communicate rapidly changing and graded information from two sources simultaneously and with minimal cross-talk. Second, we show that this multiplexing can optimize the information transferred per action potential when bursts are relatively rare. Finally, we show that neurons can demultiplex these two streams of information. We propose that this multiplexing may be particularly important in hierarchical communication where bottom–up and top–down information must be distinguished.

5 October 2012 in Science
Verified
Journal Article
Closed access
PDF file
WGWulfram Gerstner
Wulfram Gerstner
HSHenning Sprekeler
Henning Sprekeler
GDGustavo Deco
Gustavo Deco
Theory Drives Understanding: The role of theory and simulation in neuroscience has been hotly debated over the past few years, in particular in the context of the recent launch of several big projects aimed at creating artificial or virtual brains. Gerstner et al. (p. 60 ) review how theory and simulations have interacted over the years and how they have contributed to our present view of how the brain works.