CS4444: Project 4 - Organisms II

Group 3 Final Report

Mark Ayzenshtat, Ludvig Ungewitter, Yuan Zheng

1. Overview

The Organisms game challenged groups to author virtual creatures that would survive and thrive in a grid world where locomotion, consumption, and reproduction were all governed by relatively simple rules. Creatures could only move in the four major orthogonal directions, for instance, and expended a predetermined amount of energy to do so. Despite this simplicity, it became apparent that numerous strategies could prove successful under different world conditions, and that these strategies, when paired off against one another, could produce fairly complex and interesting results.

From our first player prototypes through the final tournament, we followed a few key design guidelines that we feel paid off strongly in the end. We shunned complexity wherever possible. Early in class discussions about the problem, several groups showed interest in developing a cooperative "farming" strategy for their player. We immediately recognized that such a strategy, while conceptually elegant, would probably be difficult to implement given the coordination requirements for such a system and the small inter-organism communication bandwidth that seemed to preclude it. Also, we sought, whenever possible, to make use of emergent behavior. Under most world conditions, it was virtually assured that a fair number of creatures would eventually be moving around the board, and that seemingly trivial decisions made by individual creatures could easily coalesce into a winning (or losing) strategy for the population as a whole.

For reasons of interest, we chose to focus mainly on the second stated goal of the project: competitive multiplayer worlds. Under reasonable tournament conditions, our final player fares extremely well. Our organisms eat greedily, tell friendly neighbors about adjacent food, and evolve fundamental parameters such as the rates of movement and reproduction. These strategies will be developed in greater detail in the remainder of this paper.

2. Identifying Organisms Securely

Each creature reveals an 8-bit "external state" to its neighbors. This state is the only way that one creature can communicate information to another that is not its offspring. One expects this information to be somehow valuable to at least one of the communicating parties; perhaps they are sharing the location of food or the value of a fundamental world parameter that one of them has estimated. Therefore, in such an exchange, organisms would like to be able to ascertain the identities of their neighbors. Unfortunately, it is all to easy for an enemy organism to read a neighbor's state during one turn and then mimic that state during subsequent turns, fooling other organisms belonging to that player. We developed a cryptographically strong scheme which makes such "spoofing" and other state manipulation tricks prohibitively difficult to carry out.

We refer to the very first organism in a population as the "master". When the master is born, it generates a random s-bit number, where s typically lies between 16 and 32. This number is called the seed, or h0. Then, the master applies a cryptographic hash function to h0 and strips the result of all but the first s bits, yielding a result, h1. This process is then repeated a fixed number of times, n; h(i), hashed, produces h(i+1). The entire sequence of hash values, from h0 to h(n - 1), forms a "hash chain."

The master uses this hash chain to assign its d identification bits as follows. On the first round, it sets its ID bits to the first d bits of h(n - 1). On the second round, it sets its ID bits to the first d bits of h(n - 2). On the ith round, then, it sets its ID bits to the first d bits of h(n - i). Since a cryptographic hash function cannot be inverted to produce input from output, enemy organisms are unable to determine, given a neighbor's ID at some round, what the value of that ID will be at some future round. On the nth round of the game, an organism can simply wrap around to the beginning of the chain. When the master reproduces, it passes a copy of its seed and the round number to its offspring. The newly born organisms can then rebuild an identical hash chain and carry out the exact same procedure.

Although at least two other groups utilized this secure hash chain scheme in their players, we did not. We realized that we did not actually need to implement this mechanism ourselves in order to achieve our goal of keeping other groups from pursuing spoofing and similar trickery. Ever since this idea was made public, other groups did not seem to actively develop spoofing because they were aware that this scheme made it difficult or impossible. Therefore, we got away with only "threatening" a defense, so to speak, instead of implementing one.

3. Communicating Location of Food

Each individual organism has no knowledge of the wider space beyond its four neighboring cells. Improving an organism’s knowledge of board space beyond its neighbors can greatly improve the performance of the organism. There are several pieces of information an organism can pass: presence/lack of food and presence of enemies/friends. However, the bandwidth of information an organism can pass is limited. My group reserves the 32 bits of possible parent child information passing for evolving purpose. Hence, we have only 8 bits of external state for board information passing. We decided to pass only the presence of food because of the extremely limited bandwidth.

Whether an organism finds enough food to survive and reproduce is a key in this project. Our player has a very simple strategy that if a cell has food, it occupies it and eats up all the food. Consequently, a large portion of time, our organisms stay in a cell. Those organisms do have knowledge of neighbor cells, but those information is wasted since organisms stay put no matter what happened in the neighbor cells. We therefore program to utilize those organisms and ask them to advertise their neighbor cells’ information—whether there is food in empty neighbor cells. We used four bits of external state for food locations. If in one specific direction (north, south, west, or east) there is food and no organism, the bit is set to be 1. Otherwise it is 0.

The left four bits are used for identification purpose because in order to correctly translate the advertised information, we have to be able to identify whether an organism is a friend or an enemy first.

During the game, if an organism A identifies that a neighbor organism B is a friend, A then translates the food location information advertised by B. If there is an empty cell with food next to B, A always finds the shortest path to reach that cell unless A’s energy will be exhausted before it reaches the cell. In that case, A stays put.

This strategy has one major drawback is that if the empty food cell is on the left of B, and A is on the right. A has to make 4 moves (north, west, west, south) reaching the cell. During the four moves, it is very possible that the cell is occupied by other organisms. And if the moving cost is high, A might not bother to move towards that cell at all. Even though food location is advertised, it is wasted.

We improved the strategy by change our simple “find food and eat it all” strategy. The improved organism does not hold a cell with food forever. An organism A which is sitting on food checks whether passing by organisms are friends or not. If the passing by friend organism B does not occupy a cell with food, and A knows an empty neighbor cell with food, A leaves for the neighbor cell. B then moves into the cell A previously occupies. This way, it take total two moves and the moving cost is divided. We also saved the four advertising bits for identification. Eight bits of external state is used completely for identification which greatly reduced possible spoofing.

One argument against food communication might be that an organism can reproduce to the neighbor food cell and occupy it. However, since an organism does not consider the food left in its cell and does not know how much food exists in the neighbor cell, this reproduction can lead to the dramatically decreasing of parent’s energy level. If the food left in the cell is not sufficient, it can lead to the death of the parent. Similarly, if food in the neighbor cell is not sufficient, the child organism may not survive either. Therefore, we adopted the food communication strategy and leave the reproduction decision to be made purely depends on the energy level of an organism.

4. Evolution/Mutation of Fundamental Parameters

One of our most important ideas in developing our player is the evolution idea. Our organisms evolve between generations and within generations. Those which survived are the most fit to the current environment. In different environment (in our game, different q, p, u, v, x, and y) a final survival organism’s behavior can vary dramatically. For example, in a game with a lot of food, the organism should reproduce more. In a board with a lot of organisms (both friends and enemies), the organism should move less.

There are two types of genes evolved in our organism: moving gene (a.k.a. moving factor) and reproducing gene (a.k.a. reproducing factor). Both are key decisions an organism has to make during each turn. Moving factor decides whether an organism should stay put or move towards a direction. Reproducing factor decides when (at which energy level) an organism should reproduce. During each turn, an organism checks whether its energy level is higher than maximum energy level times the reproducing factor. If so, it reproduces.

Both Factors adopt similar evolving strategies. They start with a default median value (during our implementation, always 0.5), and increment or decrement a certain amount depending the environment. The ranges of the two factors are different. Moving factor varies from 0 to 1. Closer to 0, an organism is likely to move. Whereas reproducing factor varies from 0.2 to 0.95. The smaller the number, the higher the rate of reproduction.

4.1. Intergenerational Evolving

At each reproduction, an organism generates a random number between 0 and 2. If it is 0, the organism increments its inherited moving factor. If it is 1, the moving factor in decremented. If it is 2, it stays. The same idea applies to the reproducing factor.

As the game evolves, organisms will tend to progress towards values that better fit the world conditions. For example, in an environment suitable for high values of moving and reproducing factors, more organisms with higher factors are surviving. As time goes, the organisms that have “better” values will survive at a higher rate, and slowly the population will stabilize.

The process is, as expected, successful but slow. Because it is a random process, many organisms have to be sacrificed. Our player will not know a head of time what values that fit the world. Just like in evolution, in general, some organisms take the wrong path and won’t survive. The slowness is critical because a game in the simulator enters a few different stages. It takes time for any organism to find its cycle and especially in the beginning it is a race towards easting and the population is very likely to overeat. Because of this race and mass eating, their will also be a starving period just after. Particularly, in multiplayer games, there were times were our player couldn’t survive because it evolved too slowly. Therefore, we had to integrate intra-generational evolving as a compliment.

4.2. Intra-generational Evolving

Within each generation, an organism watches its neighboring cells, and learns. If an organism see a lot of food (when more than 2 neighbor cells have food), it decrements its reproducing factor, to reproduce more. If an organism sees a very crowded board (more than 2 neighbor cells are occupied), it increments reproducing factor. It is not trivial if you want to move more or less judging from the surrounding squares. For example, in harsh climate world, with a small possibility (low p) that food “blow in”, it usually is better to move more even though the cost for moving is higher than staying put. This is because if you don’t move you will eat food as soon as it blows in and the food didn’t have time to grow (q > p). When you walk you have a better chance of finding food that has grown for a while and you can eat all that food. It is also dependent on the amount of energy per unit of the food weather the organism should walk or stay out. Additionally, when surrounding food is high, it may very well be good walk much because the population will spread faster.

This learning is rather rapid and radical but has its disadvantage being localized. An organism’s neighboring cells may not necessary represent the entire board environment and an imperfect decision can be taken. The decision will in the other hand have a good chance to be corrected rather quickly a few rounds later.

We do believe that using purely intergenerational evolvement would, over time, better match the correct configurations for a given world. Especially since the organisms quite drastically and easily can decide to chance its values. However, as said intergenerational suffers from its slowness and therefore intra-generational has to be added as a compliment.

There is a great advantage, in multiplayer games, with the radical decisions that our organism takes intra-generational. At the early race of the game, many organisms are reproduced into locations were two surrounding squares contain food. Therefore, our player often takes the decision to lower reproduction factor and the effect is very quick spread. Of course, our player will suffer from this overeating but the point is that so will the competing players. Some of them may die early and many others will start out with a smaller population than they could have had. As our population blooms, most of our organisms will for a time see at least to other organisms at the neighboring squares and therefore very quickly turn conservative.

5. Gameplay Decisions

5.1. Moving

When a new organism in Group3Player5 is created it is assigned a preferred direction to walk towards. All directions, north, east, west and south, have an equal chance to be selected. The idea is that if all organisms keep just one direction they will be able to spread out as fast as possible. The organisms keep this direction until they die. However, there are two occasions when the organism chooses not to follow its decided direction. The first one is when an organism sees food. When that happens, eating the food has higher priority than the preferred direction and the organism will choose to walk towards the square that contains food. The second one is if another organism, friend or enemy, is standing in the way. In that case, our organism will choose to walk in another direction. The preferred will not change throughout the lifetime.

Since Group3Player5 is an organism that believes in evolution, it has a threshold value that decides when to walk and when to stand still. The threshold value starts at 0.5 and for every reproduction there is a chance of either an increase or decease. The value spans between 0 and 1. The organism also learns over time and then the threshold value is adjusted as discussed in section 4.1. To decide when to move, the organism generates, at each turn, a random value between 0 and 1. If this value is bigger than the threshold moving value, the organism decides to walk. If the value is smaller, it stays put. Therefore, a smaller threshold value results in more movements.

5.2. Reproducing

Just like moving, reproducing is decided based on a threshold value. It starts out at 0.5 and is modified both inter and intra generational, as discussed in section 4. The reproduction value spans between 0.2 and 0.95, it is bound because values less than 0.2 very easily leads to starvation and values above 0.95 makes the organism a bit too conservative. At each turn, a random value between 0 and 1 is generated. If that value is bigger than the reproducing value then the organism decides to reproduce. If it is smaller it will not reproduce. When it reproduces it will first try to reproduce into a square that contains food. That way the “born” organism can start eating immediately. If no surrounding squares contain food, the organism will decide to reproduce clockwise one square from its own preferred direction. For example, if the organism walks north, it will reproduce into a square to the east. It reproduces that way because it wants to prevent that two organisms are competing on the same path. Something that may very well happen if an organism would reproduce into a square aligned with its own preferred direction.

5.3. Eating

Our eating strategy is quite simple. If we see food we eat it. There is not much room for variation here. There is the option to wait next to food and let it grow. That way (if q > p) you might get out more of the food but the risk of having someone else finding the food and eating it before you is too big and you are better off in the long-run to always eat the food right when you see it.

6. Official Tournament Results

We feel that the official tournament is flawed by design and does not fairly show the performance of our group's player (or any other's for that matter). For multiplayer games, which we focused our efforts on, nearly all runs of the tournament utilize values of p and q that are well below those values typically tested in and out of class during our work on this project.

For instance, we rarely set p below 0.001 when testing players in class since this made survival so difficult. Yet, during the final tournament, 12 out of 14 runs for a given configuration feature a p that is below 0.001. As a result, the tournament results are not very meaningful for at least three reasons. Firstly, groups that optimized their players for harsh conditions had a tremendous advantage over those that optimized their players for average or abundant conditions. Secondly, a majority of the runs for a given configuration were redundant; players that failed to survive when p=0.0000324820 also tended not to survive when p=0.0000316894. As we discussed in class, when conditions are harsh, most players tend to survive only if they are fortunate enough to find a critical amount of food very early in the game. Finally, under certain configurations, a majority of the runs resulted in all of the participating players dying out.

A more useful multiplayer tournament, in our view, would have concentrated the majority of runs between p=0.01 and p=0.001 and similarly familiar ranges for the values of other game parameters. In light of the defects of the official tournament, we will not analyze its results. Instead, we ran an unofficial mini-tournament with parameters that we believed were more reasonable; we suspect that most other groups will do the same.

7. Unofficial Tournament Results

Since the official tournament results were only released two days before the paper submission deadline, and we were not expecting to have to run our own tournament, our unofficial tournament analysis is necessarily brief. Additions and bug fixes that we made before the final player deadline significantly increased our player's performance in a host of different playing conditions.

In this section, we will pictorially demonstrate the performance of our player for "reasonable" values of p and q as were used in class during most of our work on this project. Please pay attention only to the graph, which presents the performance of our player over the course of an entire game.

55x93, p=0.01, q=0.01, v=10, u=20, M=500, K=57

55x93, p=0.01, q=0.02, v=10, u=20, M=500, K=57

55x93, p=0.01, q=0.02, v=10, u=100, M=500, K=57

55x93, p=0.005, q=0.02, v=10, u=100, M=500, K=57

As shown, our player fared very well under a variety of configurations.

8. Acknowledgements and Contributions

CoreStreet's VToken certificate validation system inspired our mechanism for secure organism identification. (http://www.corestreet.com/tech-vtokens.html)

We used Group 1's "power grapher" to visually show the performance of our player in the unofficial tournament results given above.

We expect to be acknowledged for the following contributions wherever they have been used:

1. The secure organism identification scheme.

2. The evolution or mutation of individual game constants over the course of a game.

3. The use of external state to communicate the availability and location of adjacent food to friendly neighbors.

The novelty of these contributions is documented in transcripts of class discussions, archived source code submissions, and online discussion board posts.