Social institutions often use rewards and penalties to promote cooperation. Providing incentives tends to be costly, so it is important to find effective and efficient policies for the combined use of rewards and penalties. Most studies of cooperation, however, have addressed rewarding and punishing in isolation and have focused on peer-to-peer sanctioning as opposed to institutional sanctioning. Here, we demonstrate that an institutional sanctioning policy we call ‘first carrot, then stick’ is unexpectedly successful in promoting cooperation. The policy switches the incentive from rewarding to punishing when the frequency of cooperators exceeds a threshold. We find that this policy establishes and recovers full cooperation at lower cost and under a wider range of conditions than either rewards or penalties alone, in both well-mixed and spatial populations. In particular, the spatial dynamics of cooperation make it evident how punishment acts as a ‘booster stage’ that capitalizes on and amplifies the pro-social effects of rewarding. Together, our results show that the adaptive hybridization of incentives offers the ‘best of both worlds’ by combining the effectiveness of rewarding in establishing cooperation with the effectiveness of punishing in recovering it, thereby providing a surprisingly inexpensive and widely applicable method of promoting cooperation.

Self-interest frequently causes individuals engaged in joint enterprises to choose actions that are counterproductive. Free-riders can invade a society of cooperators, causing a tragedy of the commons. Such social dilemmas can be overcome by positive or negative incentives. Even though an incentive-providing institution may protect a cooperative society from invasion by free-riders, it cannot always convert a society of free-riders to cooperation. In the latter case, both norms, cooperation and defection, are stable: To avoid a collapse to full defection, cooperators must be sufficiently numerous initially. A society of free-riders is then caught in a social trap, and the institution is unable to provide an escape, except at a high, possibly prohibitive cost. Here, we analyze the interplay of (a) incentives provided by institutions and (b) the effects of voluntary participation. We show that this combination fundamentally improves the efficiency of incentives. In particular, optional participation allows institutions punishing free-riders to overcome the social dilemma at a much lower cost, and to promote a globally stable regime of cooperation. This removes the social trap and implies that whenever a society of cooperators cannot be invaded by free-riders, it will necessarily become established in the long run, through social learning, irrespective of the initial number of cooperators. We also demonstrate that punishing provides a “lighter touch” than rewarding, guaranteeing full cooperation at considerably lower cost.

The exclusion of freeriders from common privileges or public acceptance is widely found in the real world. Current models on the evolution of cooperation with incentives mostly assume peer sanctioning, whereby a punisher imposes penalties on freeriders at a cost to itself. It is well known that such costly punishment has two substantial difficulties. First, a rare punishing cooperator barely subverts the asocial society of freeriders, and second, natural selection often eliminates punishing cooperators in the presence of non-punishing cooperators (namely, ‘second-order’ freeriders). We present a game-theoretical model of social exclusion in which a punishing cooperator can exclude freeriders from benefit sharing. We show that such social exclusion can overcome the above-mentioned difficulties even if it is costly and stochastic. The results do not require a genetic relationship, repeated interaction, reputation or group selection. Instead, only a limited number of freeriders are required to prevent the second-order freeriders from eroding the social immune system.

Monitoring with implicated punishment is common in human societies to avert freeriding on common goods. But is it effective in promoting public cooperation? We show that the introduction of monitoring and implicated punishment is indeed effective, as it transforms the public goods game to a coordination game, thus rendering cooperation viable in infinite and finite well-mixed populations. We also show that the addition of within-group enforcement further promotes the evolution of public cooperation. However, although the group size in this context has nonlinear effects on collective action, an intermediate group size is least conductive to cooperative behaviour. This contradicts recent field observations, where an intermediate group size was declared optimal with the conjecture that group-size effects and within-group enforcement are responsible. Our theoretical research thus clarifies key aspects of monitoring with implicated punishment in human societies and additionally, it reveals fundamental group-size effects that facilitate prosocial collective action.

Punishment is a popular tool when governing commons in situations where free riders would otherwise take over. It is well known that sanctioning systems, such as the police and courts, are costly and thus can suffer from those who free ride on other's efforts to maintain the sanctioning systems (second-order free riders). Previous game-theory studies showed that if populations are very large, pool punishment rarely emerges in public good games, even when participation is optional, because of second-order free riders. Here we show that a matching fund for rewarding cooperation leads to the emergence of pool punishment, despite the presence of second-order free riders. We demonstrate that reward funds can pave the way for a transition from a population of free riders to a population of pool punishers. A key factor in promoting the transition is also to reward those who contribute to pool punishment, yet not abstaining from participation. Reward funds eventually vanish in raising pool punishment, which is sustainable by punishing the second-order free riders. This suggests that considering the interdependence of reward and punishment may help to better understand the origins and transitions of social norms and institutions.