Evolution of cooperation and competition can appear when multiple adaptive agents share a biological, social, or technological niche. In the present work we study how cooperation and competition emerge between autonomous agents that learn by reinforcement while using only their raw visual input as the state representation. In particular, we extend the Deep Q-Learning framework to multiagent environments to investigate the interaction between two learning agents in the well-known video game Pong. By manipulating the classical rewarding scheme of Pong we show how competitive and collaborative behaviors emerge. We also describe the progression from competitive to collaborative behavior when the incentive to cooperate is increased. Finally we show how learning by playing against another adaptive agent, instead of against a hard-wired algorithm, results in more robust strategies. The present work shows that Deep Q-Networks can become a useful tool for studying decentralized learning of multiagent systems coping with high-dimensional environments.
Solving the problem of consciousness remains one of the biggest challenges in modern science. One key step towards understanding consciousness is to empirically narrow down neural processes associated with the subjective experience of a particular content. To unravel these neural correlates of consciousness (NCC) a common scientific strategy is to compare perceptual conditions in which consciousness of a particular content is present with those in which it is absent, and to determine differences in measures of brain activity (the so called "contrastive analysis"). However, this comparison appears not to reveal exclusively the NCC, as the NCC proper can be confounded with prerequisites for and consequences of conscious processing of the particular content. This implies that previous results cannot be unequivocally interpreted as reflecting the neural correlates of conscious experience. Here we review evidence supporting this conjecture and suggest experimental strategies to untangle the NCC from the prerequisites and consequences of conscious experience in order to further develop the otherwise valid and valuable contrastive methodology.
Recent breakthroughs in neurobiology indicate that the time is ripe to understand how cellular-level mechanisms are related to conscious experience. Here, we highlight the biophysical properties of pyramidal cells, which allow them to act as gates that control the evolution of global activation patterns. In conscious states, this cellular mechanism enables complex sustained dynamics within the thalamocortical system, whereas during unconscious states, such signal propagation is prohibited. We suggest that the hallmark of conscious processing is the flexible integration of bottom-up and top-down data streams at the cellular level. This cellular integration mechanism provides the foundation for Dendritic Information Theory, a novel neurobiological theory of consciousness.
30 August 2019 in Frontiers in Systems Neuroscience
One fundamental feature of consciousness is that the contents of consciousness depend on the state of consciousness. Here, we propose an answer to why this is so: both the state and the contents of consciousness depend on the activity of cortical layer 5 pyramidal (L5p) neurons. These neurons affect both cortical and thalamic processing, hence coupling the cortico-cortical and thalamo-cortical loops with each other. Functionally this coupling corresponds to the coupling between the state and the contents of consciousness. Together the cortico-cortical and thalamo-cortical loops form a thalamo-cortical broadcasting system, where the L5p cells are the central elements. This perspective makes one quite specific prediction: cortical processing that does not include L5p neurons will be unconscious. More generally, the present perspective suggests that L5p neurons have a central role in the mechanisms underlying consciousness.
Recent advances in the field of artificial intelligence have revealed principles about neural processing, in particular about vision. Previous work demonstrated a direct correspondence between the hierarchy of the human visual areas and layers of deep convolutional neural networks (DCNN) trained on visual object recognition. We use DCNN to investigate which frequency bands correlate with feature transformations of increasing complexity along the ventral visual pathway. By capitalizing on intracranial depth recordings from 100 patients we assess the alignment between the DCNN and signals at different frequency bands. We find that gamma activity (30–70 Hz) matches the increasing complexity of visual feature representations in DCNN. These findings show that the activity of the DCNN captures the essential characteristics of biological object recognition not only in space and time, but also in the frequency domain. These results demonstrate the potential that artificial intelligence algorithms have in advancing our understanding of the brain.