Benefits in long-term memory retention and generalization have been shown to be related to sleep-dependent processes, which correlate with neural oscillations as measured by changes in electric potential. The specificity and causal role of these oscillations, however, are still poorly understood. Here, we investigated the potential for augmenting endogenous Slow-Wave (SW) oscillations in humans with closed-loop transcranial alternating current stimulation (tACS) with an aim towards enhancing the consolidation of recent experiences into long-term memory. Sixteen (three female) participants were trained pre-sleep on a Target Detection task identifying targets hidden in complex visual scenes. During post-training sleep closed-loop SW detection and stimulation was used to deliver tACS matching the phase and frequency of the dominant oscillation in the range of 0.5 to 1.2 Hz. Changes in performance were assessed the following day using test images that were identical to the training (‘Repeated9), and images generated from training scenes but with novel viewpoints (‘Generalized9). Results showed that active SW tACS during sleep enhanced the post vs. pre-sleep target detection accuracy for the Generalized images compared to sham nights, while no significant change was found for Repeated images. Using a frequency agnostic clustering approach sensitive to stimulation-induced spectral power changes in scalp EEG, this behavioral enhancement significantly correlated with both a post-stimulation increase and subsequent decrease in measured spectral power within the SW band, which in turn showed increased coupling with spindle amplitude. These results suggest that augmenting endogenous SW oscillations can enhance consolidation by specifically improving generalization over recognition or cued recall. SIGNIFICANCE STATEMENT This human study demonstrates the use of a closed-loop noninvasive brain stimulation method to enhance endogenous neural oscillations during sleep with the effect of improving consolidation of recent experiences into long-term memory. Here we show that transient slow oscillatory transcranial Alternating Current Stimulation (tACS) triggered by endogenous slow oscillations and matching their frequency and phase can increase slow-wave power and coupling with spindles. Further this increase correlates with over night improvements in generalization of recent training to facilitate performance in a target detection task. We also provide novel evidence for a tACS induced refractory period following the tACS induced increase. Here slow-wave power is temporarily reduced relative to sham stimulation, which nonetheless maintains a positive relationship with behavioral improvements.

Sleep is critically important to consolidate information learned throughout the day. Slow-wave sleep (SWS) serves to consolidate declarative memories, a process previously modulated with open-loop non-invasive electrical stimulation, though not always effectively. These failures to replicate could be explained by the fact that stimulation has only been performed in open-loop, as opposed to closed-loop where phase and frequency of the endogenous slow-wave oscillations (SWOs) are matched for optimal timing. The current study investigated the effects of closed-loop transcranial Alternating Current Stimulation (tACS) targeting SWOs during sleep on memory consolidation. 21 participants took part in a three-night, counterbalanced, randomized, single-blind, within-subjects study, investigating performance changes (correct rate and F1 score) on images in a target detection task over 24 h. During sleep, 1.5 mA closed-loop tACS was delivered in phase over electrodes at F3 and F4 and 180° out of phase over electrodes at bilateral mastoids at the frequency (range 0.5–1.2 Hz) and phase of ongoing SWOs for a duration of 5 cycles in each discrete event throughout the night. Data were analyzed in a repeated measures ANOVA framework, and results show that verum stimulation improved post-sleep performance specifically on generalized versions of images used in training at both morning and afternoon tests compared to sham, suggesting the facilitation of schematization of information, but not of rote, veridical recall. We also found a surprising inverted U-shaped dose effect of sleep tACS, which is interpreted in terms of tACS-induced faciliatory and subsequent refractory dynamics of SWO power in scalp EEG. This is the first study showing a selective modulation of long-term memory generalization using a novel closed-loop tACS approach, which holds great potential for both healthy and neuropsychiatric populations.

Background: Poor sleep quality is a common complaint, affecting over one third of people in the United States. While sleep quality is thought to be related to slow-wave sleep (SWS), there has been little investigation to address whether modulating slow-wave oscillations (SWOs) that characterize SWS could impact sleep quality. Here we examined whether closed-loop transcranial alternating current stimulation (CL-tACS) applied during sleep impacts sleep quality and efficiency. Methods: CL-tACS was used in 21 participants delivered at the same frequency and in phase with endogenous SWOs during sleep. Sleep quality was assessed in the morning following either verum or sham control stimulation during sleep, with order counterbalanced within participants. Results: Higher sleep quality and efficiency were found after verum stimulation nights compared to control. The largest effects on sleep quality were found immediately following an adaptation night in the laboratory for which sleep quality was reduced. Conclusions: Applying CL-tACS at the same frequency and phase as endogenous SWOs may offer a novel method to improve subjective sleep quality after a night with poor quality sleep. CL-tACS might be helpful for increasing sleep quality and efficiency in otherwise healthy people, and in patients with clinical disorders that involve sleep deficits.

In this article, we consider a subclass of partially observable Markov decision process (POMDP) problems which we termed confounding POMDPs. In these types of POMDPs, temporal difference (TD)-based reinforcement learning (RL) algorithms struggle, as TD error cannot be easily derived from observations. We solve these types of problems using a new bio-inspired neural architecture that combines a modulated Hebbian network (MOHN) with deep Q-network (DQN), which we call modulated Hebbian plus Q-network architecture (MOHQA). The key idea is to use a Hebbian network with rarely correlated bio-inspired neural traces to bridge temporal delays between actions and rewards when confounding observations and sparse rewards result in inaccurate TD errors. In MOHQA, DQN learns low-level features and control, while the MOHN contributes to high-level decisions by associating rewards with past states and actions. Thus, the proposed architecture combines two modules with significantly different learning algorithms, a Hebbian associative network and a classical DQN pipeline, exploiting the advantages of both. Simulations on a set of POMDPs and on the Malmo environment show that the proposed algorithm improved DQN’s results and even outperformed control tests with advantage-actor critic (A2C), quantile regression DQN with long short-term memory (QRDQN + LSTM), Monte Carlo policy gradient (REINFORCE), and aggregated memory for reinforcement learning (AMRL) algorithms on most difficult POMDPs with confounding stimuli and sparse rewards.