Purple Pestilence
Group 6 – COMS 4444
29. September 2004
Akash Garg
Bhagyashree Bohra
John Cieslewicz
Introduction
The goal we attempted to achieve was quite obviously to have our organisms outlast the competition in well-nourished and scarce environments. Throughout the development for the optimal strategy, we have undergone many changes in ideas and ways to approach the problem. We tried many methods that seemed like a good idea at the time but had to be disregarded because it did not work. Much of the development of new strategies came from observing how our organisms competed with those of other groups in the class during the test runs. Throughout much analysis, we believe we have developed a decent strategy that will last in any kind of environment and it is to our surprise much simpler than we had originally thought.
The remainder of this document will describe the approaches we have taken throughout the development of our strategy and how we evolved from one strategy to another to come up with our final strategy.
Phase One – Black Plague
During a test run in class from last year’s players we decided to adapt Group 5’s strategy [Black Plague]. They had a very simple and very effective strategy and introduced the life-cycle element into the game. They would keep track of the each organism’s age (the number of turns they have been alive). Each organisms age would determine how much they move and how much to reproduce. The older the organisms are the more aggressive they become. The end result is, during the beginning stages of the game, there is small growth and a very small number of organisms since they are very young and play conservatively. However, during later stages of the game, the organisms become more and more aggressive and expand to take over the board. This is a very effective strategy that outlasts the competition in most environments.
We began by analyzing Black Plague to discover flaws in the strategy and try to fill any voids left behind. We initially thought that the player could be improved significantly by providing it with better information about its environment such as improved measures of food and population density. We also sought ways to eliminate the numerous constant thresholds in Black Plague—although it did progress through a lifecycle, the lifecycle thresholds were based on constants more than life experience and environment conditions. Our approach was to initially decide what our organism could “know” about the environment and its experience.
Phase Two – Improving Black Plague
After familiarizing ourselves with the Black Plague, our next task was to improve the player such that we can defeat the black plague. This took a considerable amount of effort and the end result was not what we had expected. There were many different aspects we explored and did not attempt, each of which is listed in the following section:
Estimating P and Q
During many class discussions, few groups decided that they were going to spend some amount of their time in order to estimate P and Q and then determine their strategy based on their estimate. Although, this is a perfectly valid strategy, we determined that it would not be a good idea. First, we are wasting valuable time at the beginning stages of the game to estimate P and Q when we could be exploring, eating, reproducing and increasing our population. Furthermore, knowing the information for P and Q we thought was not all that beneficial to the strategy that we are going to choose because in a reasonable competitive environment every player consumes most if not all of the food on the map relatively all the time. So at the beginning stages of the game we see an increase in food but after organisms start consuming food and reproducing the amount of food on the board stays at a very low level, as can be seen from Figure 1 below. Thus, P and Q do not really play an important role in these situations.
Figure 1 Food drops off after initial phase of the game.
Farming
Many groups from last year and this year determined that farming was a good strategy. However, after analyzing competitive environments, we again decided that this was not a good idea. In a competitive environment organisms tend to consume all the food available on the board. Thus, there is no food available on the board that could support a decent sized farm. The other problem occurs when there is just not enough real-estate on the board available to make a farm. All spaces are taken up by other organisms that are also expanding leaving little to no space for a farm. The only possible farm one could build would be a Diamond Farm [Hunters and Farmers]. However, since there are too many organisms on the board that consume all the food as it shows up, it is extremely unlikely to find a single spot on the board that contains a reasonable amount of food to support four players for a good period of time. The figure below demonstrates the over population on the board and the lack of food and real-estate to build a farm (Figure 2).
Figure 2 Densely populated environment. Farming would be difficult.
Farming is a very reasonable strategy in a single player environment and when the values of P and Q are relatively high. However, our goal was to develop a multiplayer organism and thus did not consider this option any further.
Optimistic Farming
In regard to farming, we had a very simple and optimistic approach, where we decided that at the start of the game, we should reproduce until we have a population of four. Then we would right away build a Diamond Farm around a particular square on the board where we start off even if that square did not have food. If food were available we would build a farm around that.
The basic premise for this approach was that if there food grows in areas that are not inhabited, and then presumably food will grow around in the area we have built a farm around. This is done at the very beginning of the game when there aren’t many organisms on the board the food is readily available. Once we have built the farm, we wait optimistically and hope that food grows. If it does, then we are able to sustain ourselves for a very long period of time and the four original organisms can serve as reproducers since they have a conceivable infinite amount of energy feeding them, given that Q is high.
Unfortunately, this approach did not work too well and we had to abandon it. The problem was that the environments were not nearly as rich enough to create food in that particular area. There was plenty of food growing; however, it was very difficult for it to grow in the center of the farm. The other downfall was that the organisms had to reproduce right at the beginning of the game. Therefore there were four very weak organisms waiting for food to show up next to them. If food does not appear, all four would die, which was the result most of the time.
Communication
Many groups also had this notion of communication with the other organisms of the same species. This would be helpful to pass information to your neighbors like the location of any food near by, how many people there are on the board, what is the estimate of P and Q etc. Communication also plays a big part in strategies that involve farming since players must coordinate with one another into when to move in and take the food [Hunter and Farmers]. The flaws that we saw in communication schemes were the following:
- Only one byte was available for communicating with your neighbors. This is a very restricted communication medium and only a limited number of information can be passed.
- Spoofing other players that use communication is simple since the messages are broadcast to everyone around. There is no way of telling which organisms are actually your own unless an elaborate encryption mechanism is developed. However, in 8 bits, how elaborate can you get?
- Organisms can only communicate with other organisms that are directly next to them. This is very in effective we thought since there is very little information that actually gets passed along to other players. If you were able to communicate in a wider range, this would perhaps work better as you could let more organisms know the location of food for example.
Needless to say, we dropped the idea of communicating with fellow organisms. However, we did implement a communication mechanism between parents and children. This is much more effective because instead of 8 bits we are allowed to pass down 32 bits to the children. This allows for more information to be passed along to the child. Furthermore, parents and children are most likely to be located more or less in the same geographical area at least during the early stages of the child’s life. Therefore, the information that the parent contains about a local region can be effectively passed down to the child. Since there are 32 bits available, this allows for a very experienced child to be born with a good estimate of the local area around them. We chose to pass down two particularly interesting pieces of information to the offspring: Neighborhood Density and Food Density. These are explained in the following sections.
Neighborhood Density
In order to determine when we should move, our organism keeps track of a neighborhood density. This is an estimate of how dense the world is around a particular organism. The neighborhood density averages the number of organisms that surround a particular organism every turn. The values vary from 0 – 4, since at most four organisms can surround another one. This average is computed locally for every organism and then this average is passed down to the offspring once the organism reproduces.
This heuristic gives a good estimate of how populated the world is and determines if the organism should move or not. In a very populated environment, there is not much food available (see Figure 2), hence moving to find food is not usually a good option since the organisms will be wasting energy moving and not finding any food. Conversely, in a sparsely populated world, organisms are more likely to find food if there is none around them since food tends to grow when fewer organisms reside on the board.
In regards to moving, we compare our local neighborhood density and compare it with the average neighborhood density of the world we have seen so far. A densely populated environment will have little or no available food; one hypothesis is that the best strategy for an organism in a dense environment is to wait for food to appear next to it, thus reducing unproductive movement. The greatest of food appearing on a given turn is when the four cells neighboring the organism are empty. Therefore, if our local density is less than the average, we decide to stay put. Otherwise it is a better idea to move to a less dense area with the hope that we find some food. The basic algorithm is shown below:
if(currentDensity > avgDensity) then moveAway();
Food Density
Similar to neighborhood density, we also keep track of the number of food units we have seen around us. This information is also averaged over the number of moves we have made. This information is also used to determine if we are in a rich environment. Moving a very expensive and it should only be done if we are sure that we are to find food when we move. As a result simply moving to a less dense area might not be a good idea if we have not seen much food around, indicating that the value of P is low. This average is computed for every organism and like the neighborhood density it is also passed down to the offspring. This gives children a rough idea of whether there is food available in the area locally and thus determining if they should move or not whether to find food.
Black plague used a ratio of food eaten to an organism’s age to give an idea of the richness of the environment. Hence the more you have eaten at a given age, the more likely it is that your environment is rich.
We decided to adopt this new ratio to provide an estimate of the richness of the environment. We felt this may provide a better estimate of the environment richness than just the food eaten over the organism’s life span. Food eaten does not take into account the other food around you, which even though may not be eaten by you, will still suggest a richer environment.
This average gives us a very rough heuristic of what P is. Depending on the richness of the environment we move probabilistically (move with decreasing probability as environment gets scarce).
Results
In scarce environments our player had difficulty because it moved too much wasting valuable energy. Still, we were able to beat the black plague more than ½ the time.
Phase Three – Starting Over with Player N
After experimenting with Black
Plague modifications, we had a number of new ideas and decided to start a new
player from scratch. Below are the strategies we explored with this new player.
Notably, the lifecycle was replaced by a probabilistically chosen nascent and
conservative strategy.
Nascent strategy
After observing the behavior of Black Plague, we observed that initially it did not move if food didn’t show up. So in cases where there wasn’t food next to it but close by, the player’s strategy proved ineffective. Hence we felt that it may be reasonable to consider moving around initially to find food.
We decided that if the organism is newly born and is a high-energy organism, it might be worth taking the risk to go about finding food. So the organism moves around to find food till it has energy left for 15 moves. However this did not work out very well since the organism seemed to be spending a lot of energy but did not benefit as much.
So we looked for a trade-off between being conservative and just waiting for food to show up and going about looking for food aggressively.
We decided to make two types of organisms, nascent (aggressive when born) and non-nascent. When an organism is born, we randomly choose whether or not it will be a nascent player. Now the nascent organism goes looking for food till it has energy left for sixteen moves or finds food, whichever comes first. This improved the performance of the organism over the one using the previous version of the strategy. The non-nascent ones do not incorporate this strategy. This seems to give a fair balance between risk taking and being risk averse.
Though this strategy did not give a lot of benefit, we feel it has potential and may require a different way of implementation and using better numbers.
Conservative Strategy
The opposite of the nascent strategy, we also created a conservative strategy in which the organism only moves when it can move to food. Similar to the nascent strategy, we randomly choose at birth whether the organism will be a conservative player. If not conservative or nascent, the organism follows the movement rules described below.
Moving To Food
When an organism’s location does not contain food, it should move to a neighboring location with food before considering any other action[1]. Food is a scarce resource and passing an opportunity to acquire more hurts the organism while benefiting the organism that does acquire it. Therefore moving to available adjacent food should occur with higher precedence all other actions.
Average Distance to
Food
One important relationship we noticed was the relative cost of moving with respect to the amount of energy contained in one unit of food. That ratio should significantly influence an organism’s propensity to move. As a boundary case, consider the environment where the cost of moving (V) equals the amount of energy gained by eating one unit of food (U). In this case, moving to food results in no gain if only one piece of food is found. This harsh environment was not considered to be viable, but this analysis led to a more applicable result for environments in which V approached U. If the organism moves to an empty square, it will necessarily move at least once more before eating food, resulting in at least 2 moves when the organism moves speculatively to improve its chance of finding food. Strategies involving one speculative move in environments where 2V >= U will succeed only if the organism can expect to find more than one unit of food in a cell, which we find to be contrary to experience.
In late stages of the game, the organism will usually find food in units of one. After the initial game stage, during which food is usually relatively abundant, P overwhelmingly dictates the availability of food, the probability food appears in a cell. Because the environment is relatively saturated with competitive organisms, this food is normally captured before it doubles due to Q, the probability of food doubling. Therefore, we assert that in competitive environments, after the initial stage of the game food is found most often in units of one.
Given that an organism will usually find only one piece of food when it finds food, expending more energy moving to find food than it can expect to recover is unwise. Rather than estimate P and Q, we measure the average distance to food, which results in an easily applied heuristic:
if (AverageDistanceToFood < U/V){
MoveSpeculativelyWithAnOptimisticProbability();
}else{
MoveSpeculativelyWithAPessimisticProbability();
}
We move with a probabilistic chance in both cases since we want to avoid hard threshold behavior where the organism either moves or does not move. The optimistic and pessimistic probabilities are hand-tuned factors that are the likely weakness of this strategy since we did not have enough time to thoroughly tune the values.
The use of an average distance to food also has the nice property of causing the organism to transition in and out of conservative movement behavior. When the distance to food becomes too costly, the chance of the organism moving speculatively becomes small. The organism will then only be likely to move if it knows it will eat, that is, when food is available in a neighboring cell. The moves to reach this food as well as the amount of food available in that cell (probably one unit of food based on our above observations) are factored into the average distance to food, lowering that average. The lowering of the average distance to food will eventually cause it to fall below the U/V threshold, which results in a higher probability of speculative behavior. In an environment where population and food availability follow a saw tooth pattern of boom and bust, this strategy allows the organism to adapt to periods of scarcity while still taking advantages of periods of plenty. In a single player environment, our organism’s population does rise and fall periodically, but overtime it maintains roughly the same average population and total energy level.
What to do with a Cache
of Food?
An organism should never move from a supply of food. Some groups discussed using a communication scheme for yielding a food cache to a more needy organism of the same species. Our group decided that such a communication scheme would have numerous drawbacks: communication takes valuable turns, the food may inadvertently surrendered to an enemy, and sharing food may encourage organisms to clump together, thus reducing the chance that new food appears next to an organism. Instead of sharing a food cache, our organism makes decision to use that cache unilaterally for the benefit of the population at large. First, the organism has succeeded in finding food so it should be rewarded by eating, thus keeping it alive along with its collective experience, including its calculation of population density and the average distance to food, which may be quite developed. While eating, the organism may reproduce if the reproduction criteria are met, thus using this food resource to expand the organism population.
Finally, the organism cannot eat if energyleft > M-U. In this situation we force reproduction. The existence of a food cache such that this condition exists suggests that the cell contained more than one unit of food. Because of our observations discussed earlier, a cramped and competitive environment is unlikely to exhibit many cells with greater than one unit of available food. Therefore finding such a surplus suggests the converse- an environment rich enough in resources to support an expanding population; therefore we reproduce in an attempt to take advantage of a plentiful environment. This is one method of speculatively growing our population in the presence of abundant resources.
Move Without
Backtracking and Stay Away From Friends
Two last refinements to our organism’s moving strategy was to implement a “no backtracking” policy and to move away from friend organisms when we expected to find a less dense location by moving away.
The “no backtracking” policy prevents an organism from returning immediately to the previous cell. This policy helps reduce needless moving when the organism is effectively boxed in, but whose population density estimate does not yet restrain its movements. The no back tracking policy is also beneficial in environments where we hope to find food by moving speculatively. Given the passing of a number of turns, food is more likely to have arrived elsewhere if we have not observed it arriving, so it makes sense not to retrace of immediate past—exploring new territory is more likely to lead to food, especially in an environment with a lower population density.
As in the improved Black Plague code, we calculate the average population density. Instead of moving away from situations that are more crowded than the average, we check to see if any of our neighbors are friends, and if so, we move away from them, but we stay next to enemies. This choice is inspired by observations of Group 7’s successful player that successfully spread out and conquered the board, in part, by moving away from friends. To determine friend from foe, we examine our neighbors’ external state. At the beginning of the game our initial organism chooses a random one-byte number to be used by all of our organisms. When reproducing, the identifier is passed to the newborn child who sets it as its external state. Assuming no organism is spoofing, there is a small probability that our id will collide with other organisms’ external state. If an organism is spoofing external states then we will move away from that player as if it were a friend. This may be a weakness in our strategy, however we did not have to explore the consequences of spoofing on our organism.
With a strategy for identifying friendly organisms in place, implementing moving away from friends was easy and we beneficial. Since we did not incorporate cooperative behavior into our organism, there is no advantage in clumping together, but by moving away, our organisms 1) may find food and 2) create space in which food may appear. Not moving away from an enemy organism makes sense since by remaining in place our organism is depriving the enemy of both a direction of movement as well as potential access to food. Moving away costs us energy and we will not have to contend with an enemy organism should food land in the cell we vacated. This strategy is not perfect, but it did seem to help limit some enemy movement, particularly when our organism’s population comprised a majority of the total organism population in a densely populated environment.
Reproducing
After observing our player while running with others in the class, we realized that our reproduction strategy also needed some work. What we have noticed is that the game progresses in series of peaks and valleys. When food is available organisms tend to reproduce and consume the food available. Soon, there is no food remaining on the board. As a result, there is a decrease in population since organisms do not get any food to consume. Once there are less organisms on the board, space is available for food to grow again and the pattern continues from the beginning.
We needed a reproduction strategy that will take advantage of this cycle. So we disregarded the Black Plague’s age lifecycle and implemented one that takes food availability into account. Following the peaks and valleys model, we want to reproduce when during the peaks when food is available and not during valleys when food is scarce. This ensures that when organisms reproduce, their offspring survive because they are able to find food. If there is no food available, the offspring will die and effectively the organism has wasted ½ of its energy.
We restrict reproduction for each organism until the organism has found food at least once. Once an organism has found food, it is able to reproduce given that it meets one of the following criteria:
- If organism is sitting on food and there is food next to it, reproduce onto that food unit.
- If the organism’s energy meets the following inequality, then it can reproduce: (energy + foodleft * U > (3/5)*M). This inequality takes into account the energy that the organism will have once it has eaten the food it is sitting on. We never leave a square if it contains food, until it is gone. This also handles the case when the organism is not sitting on food (foodleft = 0), its current energy must be greater than 3/5 the maximum energy available.
Additionally, the reproduction capability of an organism is disabled, if the organism has lost enough energy since the last time it ate that it cannot retrieve by eating one unit of food. This case is a heuristic indicating that the organism has not eaten food in quite some time so it would not be a good choice to reproduce since there might not be food readily available for its offspring.
This reproduction strategy is a lot more aggressive than our previous strategy and during beginning stages of the game we usually swarm the board (see Figure 3) in comparison to Black Plague (Figure 4). However, as the game progresses you can see the peak and valley effect of reproduction. Nevertheless, our population seems to be decreasing slowly despite the peak and valley model.
Figure 3 Purple Pestilence in a single player game at round 100 (P=.01, Q=.02).
Figure 4 Black plague (colored blue) in a single player game at round 100 (P=.01, Q=.02).
Results
Combining the three above strategies did not change the player much. We are a little bit stronger, but not a great deal of difference can be seen. The best strategy we found was combining the Reproduction strategy from above and the moving strategy that depends on neighborhood and food densities. This combination seems to work the best and we are able to beat the black plague during our test runs 10 – 0 (see Figure 5).
Figure 5 Our Group5PlayerAG2 overwhelms Black Plague (colored blue here for demonstration purposes).
However, when running the tournament with other dominant players such as Group 7 and Group 8 the results differ[2]. Group 8 does a fantastic job in harsh environments when P < .002 and we are unable to beat them in such environments (Figure 6). However, in richer environments we usually are a better match against both Group 7 and Group 8, though our population does decline over time (Figure 7).
Figure 6 Our PlayerN vs. Group7's OrangePlayer2 "The Net" and Group 8's integWanderer "Survival of the Fittest". Group 8 dominates in the long run. (P=.002)
Figure 7 Our PlayerN vs. "The Net" and "Survival of the Fittest" in a rich environment. Our player survives, but is weakened. (P=.01, Q=.02).
Tournament Results
In the tournament we submitted two players. Group6PlayerAG2 was the modified Black Plague player and Group6PlayerN is our new player implementation, they performed similarly. The results for Player N, our overall stronger submission, are described below.
Single player game, Player N:
Our player survived in all the tournament configurations in the single player mode. We finished in the top 50% for 75% of the tournament, based on population. The population was fairly high and energy levels were good.
Our player does well in a variety of configurations. The high population suggests that our reproduction strategies are good. The energy levels are indicative of the fact that we seem to have established a fair balance between staying put to conserve energy and moving around aggressively to find food [can be expressed better!!]
Multi-Player games, Player N:
Our player survived in 75% of the tournament configurations and was in top 30% in 50% of the rounds and was in the top 50% in the remaining 25%, in terms of population. This suggests that our player is a fairly competitive player and does dominate in a majority of the cases.
Our strategies are well suited for competitive as well as single player games.
Future Work
Almost all strategies used in this class came to depend on constant thresholds or “fudge factors” to influence organism behavior. Having good information about the environment is only useful if it is applied correctly. Therefore, if we had more time we would like to take the environment information we are able to calculate, and perform offline analysis to utilize that information more effectively. By picking a particular environment niche (i.e., competitive, scarce, single player) and running automated tests using a large sample of possible constants we feel it may be possible to tune our player to dominate in a particular environment.
Conclusion
We conclude that we were successful in building a fairly robust and versatile player. We implemented some new ideas, many of them proved beneficial .Our player could have done better, probably with a change in some aspect of the implementation with a deeper mathematical analysis of the ratios and number used.
References
Black Plague (2003)
Hunters and Farmers (2003)
Group 7 (2004)