Hava Siegelmann

Prof. Dr. Hava Siegelmann


Research Focus & Metrics

Academic Career Length

36Years (1991-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
13 August 2020 in Nature Communications
Verified
Journal Article
Open access
PDF file
GVGido M. Van De Ven
Gido M. Van De Ven
HSHava T. Siegelmann
Hava T. Siegelmann
ATAndreas S. Tolias
Andreas S. Tolias
Artificial neural networks suffer from catastrophic forgetting. Unlike humans, when these networks are trained on something new, they rapidly forget what was learned before. In the brain, a mechanism thought to be important for protecting memories is the reactivation of neuronal activity patterns representing those memories. In artificial neural networks, such memory replay can be implemented as ‘generative replay’, which can successfully – and surprisingly efficiently – prevent catastrophic forgetting on toy examples even in a class-incremental learning scenario. However, scaling up generative replay to complicated problems with many tasks or complex inputs is challenging. We propose a new, brain-inspired variant of replay in which internal or hidden representations are replayed that are generated by the network’s own, context-modulated feedback connections. Our method achieves state-of-the-art performance on challenging continual learning benchmarks (e.g., class-incremental learning on CIFAR-100) without storing data, and it provides a novel model for replay in the brain.

30 August 2021 in Neural Computation
Verified
Journal Article
Closed access
PDF file
THTyler L. Hayes
Tyler L. Hayes
GKGiri P. Krishnan
Giri P. Krishnan
MBMaksim Bazhenov
Maksim Bazhenov
HSHava T. Siegelmann
Hava T. Siegelmann
TSTerrence J. Sejnowski
Terrence J. Sejnowski
CKChristopher Kanan
Christopher Kanan
Replay is the reactivation of one or more neural patterns that are similar to the activation patterns experienced during past waking experiences. Replay was first observed in biological neural networks during sleep, and it is now thought to play a critical role in memory formation, retrieval, and consolidation. Replay-like mechanisms have been incorporated in deep artificial neural networks that learn over time to avoid catastrophic forgetting of previous knowledge. Replay algorithms have been successfully used in a wide range of deep learning methods within supervised, unsupervised, and reinforcement learning paradigms. In this letter, we provide the first comprehensive comparison between replay in the mammalian brain and replay in artificial neural networks. We identify multiple aspects of biological replay that are missing in deep learning systems and hypothesize how they could be used to improve artificial neural networks.

1 November 2019 in Neural Networks
Verified
Journal Article
Closed access
DSDaniel J. Saunders
Daniel J. Saunders
DPDevdhar Patel
Devdhar Patel
HHHananel Hazan
Hananel Hazan
HSHava T. Siegelmann
Hava T. Siegelmann
RKRobert Kozma
Robert Kozma

22 March 2024 in Nature Machine Intelligence
Verified
Journal Article
Closed access
PDF file
ASAndrea Soltoggio
Andrea Soltoggio
EBEseoghene Ben-Iwhiwhu
Eseoghene Ben-Iwhiwhu
VBVladimir Braverman
Vladimir Braverman
EEEric Eaton
Eric Eaton
BEBenjamin Epstein
Benjamin Epstein
SKSoheil Kolouri
Soheil Kolouri

1 July 2020 in Artificial Intelligence in Medicine
Verified
Journal Article
Closed access
MSMark Shifrin
Mark Shifrin
HSHava Siegelmann
Hava Siegelmann

1 November 1996 in Computational Intelligence
Verified
Journal Article
Open access
PDF file
HSHava T. Siegelmann
Hava T. Siegelmann
This article studies finite size networks that consist of interconnections of synchronously evolving processors. Each processor updates its state by applying an activation function to a linear combination of the previous states of all units. We prove that any function for which the left and right limits exist and are different can be applied to the neurons to yield a network which is at least as strong computationally as a finite automaton. We conclude that if this is the power required, one may choose any of the aforementioned neurons, according to the hardware available or the learning software preferred for the particular application.