Analysis of the article

Analysis of the article

Evolution of Indirect Reciprocity

by Image Scoring [1]

by Nowak, M., Sigmund, K., Nature, 1998

Essay by: João Moreira

Tomás Aquino

Evolution of Indirect Reciprocity by Image Scoring Index

1. Introduction .................................................................................................................. 3 1.1. Motivation ............................................................................................................. 3 1.2. Model ..................................................................................................................... 3 2. Results and Discussion ................................................................................................. 5 2.1. No Mutations, Complete Information ................................................................... 5 2.2. Mutations, Complete Information ......................................................................... 7 2.3. Mutations, Incomplete Information ....................................................................... 9 2.4. Mutations, h-Strategy .......................................................................................... 10 2.4.1. AND .............................................................................................................. 10 2.4.2. OR ................................................................................................................. 11 2.4.2. Only own image score .................................................................................. 12 3. Overview .................................................................................................................... 12 4. References .................................................................................................................. 12

2

Evolution of Indirect Reciprocity by Image Scoring 1. Introduction

1.1. Motivation

The arising of cooperation in a population has been in the minds of many researchers from different fields in the past few years. The prevailing idea used to be that Darwinian “survival of the fittest” favours selfish behaviour; however, it is a well known fact of everyday life that many populations, including human societies, exhibit extensive cooperative behaviour. The goal, then, is to provide a model in the spirit of Darwinian evolution such that cooperation has a chance to win against defection, in the sense that cooperative individuals will be more successful and thus leave more offspring whose behaviour will be similar to their parents’.

1.2. Model

The authors present a model within the framework of Game Theory, each individual of the population being represented by a player with a certain strategy. A cooperative relationship has in this model a giving and a receiving end, and a player’s actions result in a certain payoff. So, when an individual decides to “help” another, it will come with a certain cost, whereas they will profit when on the receiving end of the relationship. How fit an individual is is then measured by the payoff they accumulate.

The authors consider indirect reciprocity as the mechanism behind the arising of cooperation. They argue that in order for the altruistic act to be advantageous (in the crude sense that it will make it more likely for the individual to be helped later on and so increase their fitness) it is not necessary that the helped individual will be the helper later on: other people in the population will be aware of a certain individual’s actions, and will have them in mind later on when deciding whether to help him or not – the so-called “I won’t scratch your back if you won’t scratch their backs” principle. With this idea in mind, the authors introduce an image score for each player, which represents their reputation in the population.

More concretely, each player has a strategy k5,4,...,0,...,6, and an image score s5,4,...,0,...,6. The population consists of N players and is assumed to be well mixed (everybody has the same chance of interacting with anybody else). The

3

Evolution of Indirect Reciprocity by Image Scoring assumption of “well-mixedness” can be justified by the fact that small populations (N up to 150) are used, which can be thought of as small communities where everybody knows everybody else. To begin with, an interaction between two players is witnessed by the whole population, meaning that a player’s image score is known to every player. The game starts with a random distribution of strategies, and is played for a certain number tmax of generations, each one consisting of m interactions. An interaction is processed by choosing a random pair of players, one as donor and the other as recipient; the donor helps the recipient if and only if his strategy is smaller than or equal to the recipient’s image score. The donor’s image score is then increased or decreased by one unit if they decided to cooperate or defect, respectively. Cooperation results in a cost c to the donor and a benefit b (with b>c) to the recipient; otherwise, nothing else happens. Since players with strategies equal to zero or less decide to cooperate with players who have not yet had an interaction, they are termed cooperative, whereas strategies above zero are termed defective. Players with strategy -5 are unconditional co-operators, and players with strategy 6 are unconditional defectors. Strategy 0 in turn corresponds to the most discriminating cooperators.

After the m interactions, a player’s fitness is the payoff they accumulated from all interactions in which they took part. To avoid negative payoffs, c is added in every interaction to each participant’s payoff value. Then each player leaves offspring for the next generation with a probability proportional to their fitness, with the constraint that the population size remains fixed. These offspring have the same strategy as their parents’, but it is assumed that they do not inherit their reputation and comfortable position in life and so they start out with zero image and fitness scores.

Since for each interaction two players are chosen, each player will be chosen, either as recipient or donor, on average 2m/n times in each generation. According to the authors, m is then picked so that the probability of the same pair being chosen twice is negligible, in order to guarantee that “direct reciprocity” does not have an important role. We note also that players have no memory of who they interacted with in the past. So that the game becomes more interesting, mutations are added, introducing a probability that the offspring of a player will have a random strategy instead of copying its parent’s.

As a refinement, the authors then proceed to having each interaction witnessed by a certain number of other players on average, chosen at random, instead of the whole population. In this version of the model each player assigns a certain image score to

4

Evolution of Indirect Reciprocity by Image Scoring each other player, and in each interaction the perceived image score of the participants is only updated for themselves and the witnesses of that interaction.

Finally, the game is played, both with complete and incomplete information about image score, with a new element h5,4,...,0,...,6 added to the strategy of each player, so that a strategy then consists of a pair k,h. In this version of the game, three variants are played:

1 - a donor decides to cooperate if and only if their strategy is smaller than or equal to the recipient’s image score and his own image score is smaller than h; 2 - a donor decides to cooperate if and only if their strategy is smaller than or equal to the recipient’s image score or his own image score is smaller than h; 3 - a donor decides to cooperate if and only if his own image score is smaller than h.

2. Results and Discussion

2.1. No Mutations, Complete Information

For this first variant, the authors present four graphs showing the evolution over four time steps of the frequency of strategies in the population in a simulation, shown below.

Figure 1 – Simulation of indirect reciprocity in a population of N=100 individuals, with =125 interactions per generation. m

It is also stated that the strategyk 0 becomes fixed (unique) in the population in the generation t 166

5

Evolution of Indirect Reciprocity by Image Scoring We find that this is a possible outcome of a simulation for the given parameters, but not the only one. First, we analyse, for these parameter values, the time steptf in which a given strategy becomes unique in the population. We plot the corresponding distribution below.

Figure 2 – Distribution of the time step, tf, necessary for all players to adopt the same strategy. We simulated 105 runs with parameters N=100, m=125 and tmax=500.

The above graph supports the authors’ claim that, given enough time, all the players will converge on a certain strategy. The most probable value and average of tf as calculated from the distribution above are respectively 37 and 82. We then see that

tf 166 is not the average or most probable outcome, but still a result with a representative probability of occurrence.

We then proceed to plotting the probability of each strategy becoming fixed in the population, as estimated over 105 runs:

Figure 3 – Probability of each k becoming fixed. We simulated 105 runs of m=125 interactions in a population of N=100, during tmax=500 generations.

6

Evolution of Indirect Reciprocity by Image Scoring This graph shows that k 0 is indeed the single most probable outcome, but not the overall most probable one. We note that, for these parameter values, defective strategies have an overall higher probability than cooperative strategies.

To verify the authors’ claim that a higher number of interactions, m, per generation leads to an increase in cooperation, we plotted the probability of the fixed strategy being cooperative as a function of m (for N fixed), as estimated over 105 runs:

This plot indeed shows an increase in cooperation with m, and we can see that for

Figure 4 – Probability of all players adopting a cooperative strategy as a function of m. Parameter values: N=100, tmax=500.

m 500 (which corresponds to an average of 10 interactions per player) we already have a probability close to 1 of the fixed strategy being cooperative. This increase can be qualitatively understood by noting that a higher number of interactions per individual means that players with defective strategies will lower their image score further, and so increase their loss in payoff with respect to more cooperative players, who will keep their images scores higher and thus be helped more by discriminators.

2.2. Mutations, Complete Information

Adding mutations (in the form of a probability of 0.001 that an offspring does not act like its parents), the authors plot the average strategy and the average fitness over time. We find that the plots presented correspond to the typical behaviour of a simulation, but only if we assume m 400, instead of m 300 as stated in the article. In particular, the explanation given below for the observed payoff values gives the correct values for the high plateaus and for the minima only for m 400.

Below we plot the results presented in the article (left) and a typical example taken from our own simulations (right).

7

Evolution of Indirect Reciprocity by Image Scoring

Figure 5 – On the left: Results taken from the article for a population of N=100 during 3x104 generations. The article states these results are for m=300 rounds per generation. On the right: Simulation with the same N and tmax, and m=400. The upper and lower graphs show the average strategy of the population and average payoff per individual, respectively, for each generation. Both results include mutations: there is a probability of 0.001 of an offspring choosing a random strategy instead of its parent’s.

When the average strategy is cooperative, the high average payoff plateau corresponds approximately to everybody cooperating in each interaction in which they take part. On the other hand, the minima in the average payoff, which come about for defective average strategies, correspond to everybody defecting.

Average cooperative strategies close to zero (highly discriminating) persist for longer periods, because in a population where these strategies dominate it neither pays to be a defector (nobody helps you) nor to be too cooperative (you pay too much and don’t receive more in return). Defectors cannot invade one such population directly. However, too-cooperative strategies can invade through random drift (due to the mutations, lower-k players can appear and make it profitable for other players to be more cooperative, in order to be helped by them), and then it becomes possible for defectors to invade. The authors also present the frequency distribution of strategies sampled over many generations. In this case, as shown below, we find better agreement for m 400.

8

Evolution of Indirect Reciprocity by Image Scoring

Figure 6 – On the left: frequency distribution of strategies in a population of N=100,

7

with m=300 rounds per generation sampled over 10 generations (from the article). On the right: Our computer simulation with the same parameters.

2.3. Mutations, Incomplete Information

We now turn to incomplete information (maintaining mutations). In this case the authors fix an average number of 10 witnesses, chosen at random for each interaction. Then the frequency distribution of strategies, sampled over 107 generations, is plotted for N = 20, 50 and 100. In each case the number of interactions per round is m = 10N, corresponding to a fixed average of 20 interactions per player per generation. We note here that for N = 20, this corresponds to every player interacting on average one time with every other, in each generation. This will lead to some interactions between the same players. However, we do not think this is significant.

As shown below, we find good qualitative agreement with the authors’ results; however, even averaging over 107 generations, some small differences in the quantitative results still persist over different runs. The average over time of the frequency of cooperative strategies found by the authors is, for the three cases below, respectively, 90%, 47% and 18%. We found, for the presented runs, 89%, 49% and 19%.

9

Evolution of Indirect Reciprocity by Image Scoring

Figure 7 – Indirect reciprocity with only 10 viewers, on average, per interaction. The graphs show

the frequency distribution of strategies for population sizes of N=20, N=50 and N=100 sampled over 107 generations. The number of interactions is m=10N. The upper and lower graphs are the results from the article and our simulations, respectively. They are in very good agreement.

In qualitative terms, we find, as expected, that if a smaller percentage of the population has information about the actions of other players, cooperation is less likely to persist, since it is easier for defectors to remain unnoticed and be helped by a significant part of the population.

2.4. Mutations, h-Strategy

2.4.1. AND

In this case the authors present bubble plots for the distribution of (k,h) strategies, sampled over 107 generations (the frequency of a strategy is proportional to the area of the circles). Again, we find good qualitative agreement, but some fluctuations remain especially for the less frequent strategies. Below we present the plots from the article and our results, first for complete information and then for incomplete information.

Figure 8 – On the left we show the frequency distribution for And strategies with complete information, taken from the article. On the right we have the results from a simulation with the same parameters as the article: N=100, m=500, tmax=107.

10

Evolution of Indirect Reciprocity by Image Scoring The most frequent strategy for these parameter values turns out to be (k,h) = (0,1). Strategies with k > 0 are now less successful, because, since players tend to aim for lower image scores, they are now less likely to help defectors.

It is interesting to compare these results with the simulations with incomplete information. Below, we see that with incomplete information it pays to aim for higher image scores, since a cooperative act is now seen only by a fraction of the population, as noted by the authors. However, we note that the simulations were not made using the same parameter values (we have both a different N and a different number of interactions per player per generation); we think that for comparisons to be more meaningful all simulations should have been run for the same values of N and number of interactions per player per generation.

Figure 9 – On the left we show the frequency distribution for And strategies with incomplete information, taken from the article. On the right we have the results from 7

our simulation. Parameter values N=20, m=200, tmax=10.

The authors find, for perfect and imperfect behaviour, 55% and 57% for the frequency of cooperative interaction. We find 52% and 53%, again agreeing qualitatively but showing a quantitative deviation.

2.4.2. OR

Once more the distributions of the strategies sampled over 107 generations are presented. Our simulations again qualitatively agree very well with the authors’, and again small fluctuations in the quantitative results persist, especially for the less frequent strategies. The plots for complete and incomplete information, as presented in the article and obtained from our simulations, are shown below.

11

Evolution of Indirect Reciprocity by Image Scoring Figure 10 – On the left we show the frequency distribution for Or strategies with complete and

incomplete information respectively, taken from the article. On the right we have our results. Parameters for complete information are: N=100, m=500 and tmax=107; for incomplete

7

information, N=20, m=200 and t=10. max

In this case, the or condition makes for a less discriminating behavior, and we observe societies where cooperation predominates. However, the single most frequent strategy is (k,h) = (6,-5), since defectors benefit from the reduced level of discrimination. In these cases the authors find 70% and 80% for the frequency of cooperative interactions. We find 69% and 80%. 2.4.2. Only own image score

For this case the authors state only that cooperation does not arise. We plotted the frequency distribution of strategies and found that all players adopt strategies with

h5, which means that there is no cooperation.

3. Overview

Our simulations show good qualitative agreement with the authors’ results presented in the article. In the specific case of the simulations without mutation, however, the authors only present the results of a single run, which we do not find to be representative of all the possible results.

We do not find exact quantitative agreement, and observe instead small fluctuations over different runs that persist even after an averaging over 107 generations. These do not, however, compromise the qualitative results.

4. References

[1] Nowak, M., Sigmund, K., Evolution of indirect reciprocity by image scoring, Nature 393, 1998;

[2] Brokken, F., C++ Annotations Version 7.3.0, http://cppannotations.sourceforge.net.

12