Research on the diversity, evolution and stability of cooperative behaviour has generated a considerable body of work. As concepts simplify the real world, theoretical solutions are typically also simple. Real behaviour, in contrast, is often much more diverse. Such diversity, which is increasingly acknowledged to help in stabilizing cooperative outcomes, warrants detailed research about the proximate mechanisms underlying decision-making. Our aim here is to focus on the potential role of neuroendocrine mechanisms on the regulation of the expression of cooperative behaviour in vertebrates. We first provide a brief introduction into the neuroendocrine basis of social behaviour. We then evaluate how hormones may influence known cognitive modules that are involved in decision-making processes that may lead to cooperative behaviour. Based on this evaluation, we will discuss specific examples of how hormones may contribute to the variability of cooperative behaviour at three different levels: (i) within an individual; (ii) between individuals and (iii) between species. We hope that these ideas spur increased research on the behavioural endocrinology of cooperation.

Cleaning behavior is known as a classic example of cooperation between unrelated individuals. Although much is known of the behavioral processes underlying cooperative behavior, the physiological pathways mediating cooperation remain relatively obscure. Here, we show that altering the activity of serotonin on wild cleaner wrasses Labroides dimidiatus has causal effects on both social and cooperative activities. These cleaners cooperate by removing ectoparasites from visiting “client” reef fishes but prefer to eat client mucus, which constitutes “cheating.” We found that enhancing serotonin made cleaner wrasses more motivated to engage in cleaning behavior and more likely to provide physical contact to clients (tactile stimulation) without spending more time cleaning or cheating more often. Blocking serotonin-mediated response resulted in an apparent decrease in cleaners’ cheating levels and in an increase in cleaners’ aggressiveness toward smaller conspecifics. Our results provide first evidence that serotonin is a neuromodulatory driver of cooperative behavioral activities and contribute to the understanding of neural pathways of cooperation.

In an interspecific cooperative context, individuals must be prepared to tolerate close interactive proximity to other species but also need to be able to respond to relevant social stimuli in the most appropriate manner. The neuropeptides vasopressin and oxytocin and their non-mammalian homologues have been implicated in the evolution of sociality and in the regulation of social behaviour across vertebrates. However, little is known about the underlying physiological mechanisms of interspecific cooperative interactions. In interspecific cleaning mutualisms, interactions functionally resemble most intraspecific social interactions. Here we provide the first empirical evidence that arginine vasotocin (AVT), a non-mammalian homologue of arginine vasopressin (AVP), plays a critical role as moderator of interspecific behaviour in the best studied and ubiquitous marine cleaning mutualism involving the Indo-Pacific bluestreak cleaner wrasse Labroides dimidiatus. Exogenous administration of AVT caused a substantial decrease of most interspecific cleaning activities, without similarly affecting the expression of conspecific directed behaviour, which suggests a differential effect of AVT on cleaning behaviour and not a general effect on social behaviour. Furthermore, the AVP-V1a receptor antagonist (manning compound) induced a higher likelihood for cleaners to engage in cleaning interactions and also to increase their levels of dishonesty towards clients. The present findings extend the knowledge of neuropeptide effects on social interactions beyond the study of their influence on conspecific social behaviour. Our evidence demonstrates that AVT pathways might play a pivotal role in the regulation of interspecific cooperative behaviour and conspecific social behaviour among stabilized pairs of cleaner fish. Moreover, our results suggest that the role of AVT as a neurochemical regulator of social behaviour may have been co-opted in the evolution of cooperative behaviour in an interspecific context, a hypothesis that is amenable to further testing on the potential direct central mechanism involved.