Project 4: Organisms II

• Alexis Goldstein (amg2006@columbia.edu)
• Edan Harel (edan@columbia.edu)
• David Vespe (djv@columbia.edu)

Contents:

Introduction

Our overall approach was to implement and evaluate many different strategies and take the strongest parts of each. This methodology led us to several good approaches as well as several bad ones.

In this paper, we first examine the background of the problem, where we analyze the different phases of games, as well as how the board parameters interact to create different types of boards. We then discuss the common foundation on which we built our different strategies. We then examine the strategies in detail, and move on to talk about how we tuned them to work well under a variety of circumstances. Finally we discuss the results we've observed and the conclusions we've drawn from them.

[ top ]

Background

In order to understand how to make a strong player, it is first necessary to understand how games work. Games can be broken into three phases, and it is important to understand what behavior is best for each phase. In addition, the board variables, both known and unknown, must be used to tailor each player to its particular circumstances.

The phase of the game include the growth phase, the overpopulation phase, and the steady-state phase:

Phases of the Game

1. Growth Phase

   The growth phase occurs at the beginning of the game. In the growth phase, the board is mostly empty; thus, food appears often and grows quickly. (In general, it can be said that the rate of change of the number of automata on the board is proportional to amount of available food on the board minus the number of automata currently in existence. By comparison, the rate of change of food is inversely proportional to the density of the players.)

   In the growth phase, players seek out their first food. This is a critical time, because if a team's first automaton cannot find food, the team will go extinct. Furthermore, the automata have no knowledge of the values of p and q. If p turns out to be sufficiently high, the automaton that takes a chance by wandering around will be rewarded by finding food quickly and being able to proliferate before conservative players, and generally leading to the extinction of the conservative players. However, if p turns out to be very low, the player who stays still and conserves his energy is most likely to survive long enough to find food, while the aggressive player has gone extinct. Whether an organism's initial gamble on the value of p pays off determines whether he succeeds later in the game, and there is no way to avoid this gamble.

   (It could be argued that by staying put, a player could wait a few turns to estimate p. However, in practice, a player who stays put makes 4 observations about p every round. If the value of p is .01, the expected value of the wait is 12 rounds, and this produces a vast overestimate of p. If the player wants to make a more reliable estimate of p, he must wait perhaps 400 rounds. If at that time the player concludes that p is sufficiently large that movement is appropriate, he has given a 400-turn advantage to players who assumed a high value of p from the beginning. Thus by choosing to measure p, he has implicitly assumed that p is small, and will be unable to compete if p turns out to be large.)

   Once the abundance of food is discovered, automata reproduce rapidly to consume the plentiful food that is available. The end of the growth phase is marked by a downturn in the amount of food available on the board -- this occurs when organisms have reached most parts of the board and are consuming the food that is available faster than it is reappearing.

   [ top ]

2. Overpopulation Phase

   The overpopulation phase begins when the first downturn in available food occurs. In this phase, there are necessarily more automata on the board than the food growth can support, because overcrowding prevents adequate food growth. After the food that grew in the growth phase is exhausted, the automata death rate far exceeds the birth rate.

   In this phase, some automata need to die, and the weakest die first. Survival in this time does not require automata to have a lot of energy, but only to have more than those weakest automata; once the weak ones die, the remaining automata all have an equal chance of finding the food.

   Wandering, the practice of moving around through empty squares in search of food, is expensive in this phase. Food is generally not available, and the high board density means that any new food that appears will probably be consumed by a nearby automaton, so wandering is not likely to produce more food than staying put. Furthermore, wandering has a certain energy cost associated with it, and too much wandering may cause automata to have the least energy and thus be the first to die.

   The overpopulation phase ends around u*K turns after it begins. This is about enough time that an automaton who found a pile of the maximum food to consume it all, and any automata who found less food has either been able to find new sources of food or has died out. Note that for the values used in the tournament, the overpopulation period may last anywhere from around 1,000 turns to around 10,000 turn.

   [ top ]

3. Steady-State Phase

   The final phase occurs when automata have died off in sufficient numbers that a roughly stable number of automata are left. From this point on, there are cycles of growth and overpopulation, as food recovers and is then found and eaten. However, these swings tend to be far smaller than the initial one, and are often localized to particular areas of the board. It still holds true that with each swing, the weakest automata die off, and the stronger ones are left to replace them.

   The cyclic nature of the size of the overall automata population does not necessarily hold for automata of any individual team. If, for example, one team's automata tend to have low energy at the population peak of cycles, these automata will tend to die off.

   Interestingly, even if one team's automata are not consistently at the bottom of the energy ladder, extinctions will still occur. This is analogous to the gamblers' ruin problem. The situation of gamblers' ruin is one wherein a gambler plays a perfectly fair game in a casino until either he or the casino runs out of money (in our case, the casino's bank is the other players' automata viewed together, and the player's automata comprise the gambler's money). As the gambler plays, he wins sometimes and loses sometimes, just as the players' population increases and decreases. The gamblers' ruin occurs when he runs out of money, at which time he is forced to stop playing, even though we know that if he had infinite money, he would with probability 1 recover his losses. The problem for the gambler is that the casino has much more money than he does, so with very high probability, he will run out of money before the casino does. Likewise, a player with a low number of automata will with high probability go extinct before the other automata, even if the expected value of his growth rate is 0. Furthermore, a gambler playing a game of higher variance of cost/payoff (or volatility) -- or a player who experiences greater swings in population -- is more likely to run out of money/automata sooner.

   In the steady-state phase, wandering may be good or bad, depending on board parameters.

   
   In this graph of food available on the board vs. time, the growth phase is the time when food is initially rising, the overpopulation phase is when the food falls back down almost to zero, and the steady-state phase is when the food stays fairly constant.

   [ top ]

Analysis of Board Parameters

The board parameters govern which strategies will succeed and which will fail.

1. Board Size (x and y)

   A large board size is important in three ways. First, it provides a greater chance for all teams to develop independently for a time, without competition. Second, a large board ensures a large growth phase -- if the board is densely populated quickly, the growth phase will barely have begun when overpopulation occurs. Finally, with a medium-to-high value for q, large piles of food will exist, which adds an incentive for players to find and utilize this food in an intelligent way.

   [ top ]

2. Food Growth (p and q)

   For high values of p, food tends to be evenly distributed around the board. This benefits aggressive players who move and reproduce from the beginning of the game, rather than waiting.

   For low values of p, reproduction and wandering are penalized. Luck at the start of the game generally makes the difference between success and failure. These conditions benefit conservative players who wait at the beginning of the game to observe p.

   For low values of q, piles of >1 food generally don't exist. It does not matter whether the food doubling chance is one in 10,000 or one in a billion; as q decreases, below .0001, it becomes insignificant.

   If the ratio q/p is high, this leads to high variance in food distribution around the board: over time, large piles of food exist in a wasteland of no food. On the other hand, if q/p is low, q is insignificant.

   [ top ]

3. Food Yield (u)

   If food yields a lot of energy, the board can support very dense populations, since an automaton that eats just one food can survive u more turns without food. By contrast, a low value for u means lower populations and likely a lower average lifetime for automata, as unlucky ones will not find enough food to survive.

   [ top ]

4. Ratio of Food Yield to Movement Cost (u/v)

   A higher u/v means that wandering is more profitable, because the payoff from eating makes up for many turns of wandering in search of food. If u/v is low, wandering in search of food is more expensive.

   [ top ]

5. Maximum Food Per Cell (K)

   Higher values of K mean that high-food-variance boards will have even higher variance; protecting good sources of food on such boards is more profitable than on low-K boards.

   [ top ]

Foundation

• Assumptions

  Given that we need to make a gamble about initial p and q, we assume that p will be sufficiently high that wandering is useful. In this assumption, we essentially concede all games where p<.0001, and concede a lot of games in the range in the range .0001 < p < .001. This includes single-player games as well as multi-player ones; in single-player, the optimal strategy is always to wait and observe for a while, since there is no competition. We do not do this because we want to be optimal for multi-player games.

  [ top ]

• Estimating p, q, and Board Crowdedness

  In order to better make decisions about what to do, all our automata maintain a history of certain observations.

  • Estimating p

    Every time an automaton stands still (including when it reproduces), it compares the list of adjacent squares empty of food and automata before it stayed put to the list after. If it sees that food appeared on a square that was empty on the previous turn, it records this observation; if it sees that food did not appear on such a square, it also records this. Thus every automata can estimate p as the ratio of the number of food appearances to the total number of tests. The variance of this estimate is very high when the number of observations is less than 1,000, and is still high when observations number in the thousands, because this number of observations is insufficient to provide a good estimate. However, for an estimate of the order of magnitude of p, these observations are sufficient.

  • Estimating q:

    Each time an automaton revisits a food square that it has already visited, it can make an estimate of q. Since food growth is exponential, the following formula can yield an estimate for q:
    

    (foodAfter+turnsOccupied) = foodBefore * (1 + q)^(turnsSinceSeen)
    where foodAfter is the amount of food in the second observation, foodBefore is the amount of food in the first observation, turnsOccupied is the number of turns the square is known to have been occupied, and turnsSinceSeen is the number of turns between observations. This formula is slightly incorrect in that it matters not just that the cell was occupied but at what time it was occupied. We ignore this error for simplicity.

    However, even with this formula, estimates tend to be orders of magnitude off. The biggest problem is that data about q is hard to collect: it requires moving onto, off of, and back onto food, and risking that in the intervening time another automaton will move onto that food and not move off. Thus it is only truly feasible to measure q if an automaton is a farmer. But even as a farmer, one generally visits the pile of food rarely, and the food is visited many times in between by other farmers, so the data are likely to be skewed severely.

    We did not end up using estimates for q in any of our strategies.

  • Estimating Board Crowdedness

    Board crowdedness is a measure of how densely populated the board is with automata. We attempt to measure this by looking at how many other automata a given automaton has seen in the past 1, 3, or 5 turns (sometimes weighting more recent observations more heavily). However, these measurments were often very similar for crowded boards as for uncrowded boards, so we did not use them in any of our final strategies.

  [ top ]

• Communication

  We created a complex communication scheme for passing information between automata. In the end, we found that simple encrypted communications were too insecure to be reliable, and too costly to make secure. However, we laid the groundwork that others might use for strategies that require more complex message passing.

  • Parent-To-Child Messages:

    Communications from a parent to the child on the turn of birth are special in that there are 4 bytes available, and that the communication is guaranteed to be secure. All of our strategies require that two of the four bytes be used for the turn-key, which is used for encryption (and discussed later). The other two bytes are left to the disgression of the strategy. Some strategies pass a location of food or of other importance, while others pass genetic information to allow for adaptation to varying environments.

  • Inter-Automata Messages:

    Message passing between automata suffers from a variety of problems. The most basic is that automata cannot easily authenticate one another as members of the same race to share information. Furthermore, all information is broadcast to all adjacent players, so even if one knows that the target of the communication is of the same team, other enemy automata are able to hear the message. A third problem is that messages do not contain direction information so automata do not know if they are being addressed; a fourth is that the bandwidth of communcation is very low: only one byte can be passed to all neighbors every turn. A fifth problem is that time is quantized in large segments, hindering encryption: any message that one automaton broadcasts in one time slice may be intercepted and rebroadcast in the same time slice, and it is impossible to use a time slice finer than a turn. A sixth is that a discontinuity exists at board edges, leading to difficulty differentiating an attacker whose attack is a turn behind from a friendly player who is across a board boundary.

    These challenges are far from small, and arguably make communication an impossible endeavor. However, to address most of these problems, we propose the following scheme, which we call Secure Communication Over Very Low Bandwidth Channels In Quantized Time:

    1. Every communicating party must have a shared secret key. This can be shared through automata parent-child message passing. The first automaton picks this key at random.
    2. Each turn, the shared secret key is hashed according to an algorithm common to all automata, so that it is generally different each turn.
    3. Every automaton of our team broadcasts the same message, found by hashing the secret key and a constant, to identify itself as a likely member of our team. However, broadcasting the shared message is never used as proof that the automaton is on our team.
    4. When two automata of our team want to communicate with each other, each broadcasts a random challenge.
    5. Each automaton hashes the others' challenge using the turn-key, unless the random challenge was the same as the challenge issued by the automaton. If the challenges are the same, a new challenge is selected and broadcast -- otherwise the challenger could be a member of another team masquerading as one of us and trying to learn the answer to our challenge. If two reptitions turn up the same challenge, we give up and assume the other is simply replaying our message and not actually of our team.
    6. If the challenges were successfully hashed, each automaton accepts that the other has authenticated. An automaton is not allowed to move if he is engaging in communication, and must not communicate if he is about to die; this prevents another automaton from hijacking an already-established secure communication channel.
    7. Information is now passed between the automata, one byte at a time. For security, the message is encrypted by xor'ing it with a hash of the turn-key plus a constant, so that neighboring automata cannot understand what is being communicated.
    8. When an automaton wishes to move, it must send an end of channel message to terminate the connection so that the other automaton will unauthenticate an automaton in the square that is about to be moved out of.

    We implemented the bulk of this system, but communication was sometimes hampered by off-by-one errors that we did not spend the effort to fix. Though this algorithm suffers cryptographic weaknesses in that the length of the key is small and that frequency analysis on messages may lead to the key being revealed, it is probably more than sufficient encryption for the scope of this project if the bugs are worked out. There is a certain cost of communication -- automata cannot move while they are communicating -- so it can probably be afforded only on boards where movement is expensive.

    These costs of encryption mean that most groups do not use multi-turn encryption. The alternatives to multi-turn encryption include:

    1. using half the byte broadcast for authentication and the other half for the message
    2. having all members of the team broadcast the same message on a given turn, but not passing a separate message
    3. assuming that everyone else is of the same team to begin with

    This leaves them open to certain attacks. We implemented the following replay attack, which is employed by all of our automata on all turns except where they need to communicate with one another:

    Whenever you hear a byte, you repeat it. We prefer to repeat a byte from an earlier part of the board (north then west) over a later part of the board (east then south). Furthermore, we repeat the message verbatim only 50% of the time -- the other 50% of the time, we flip one bit, selected at random, before rebroadcast.

    For alternative communication method 1, 50% of the time we rebroadcast the message that the enemy automaton sent us, which is hopefully interpreted in a hurtful way; for example, if that automaton was sending us a location of food relative to it, we will send back an incorrect location of food relative to us. Alternately, 25% of the time we rebroadcast a corrupted message that the enemy will interpret as authentic. For method 2, 50% of the time we successfully impersonate a member of that team. For method 3, we likely pass back a valid but incorrect message.

    We are unsure of how successful replay attacks were at confusing other players; however, we have observed at least one instance of an opposing player walking off a big pile of food when we replayed to that automaton the message that it previously sent to us. Thus we suspect that in a small number of cases, our communications spoofing worked to our advantage.

  [ top ]

• Mapping

  All of our automata maintain a map of every cell they know about. This map includes information about how much food is believed to be on that cell (which would use our estimate for q if it were accurate), how long ago this cell was visited and observed (with observed indicating we were next to the cell but not on it), and what automata we have seen on that cell.

  This mapping functionality includes navigation functions for moving to a certain cell, and a neat printable map for debugging.

  [ top ]

Spiral

This strategy alone does not involve a rule for when to reproduce, so in any circumstance where reproduction is sometimes desirable (essentially where v>1 and p>.00001), this strategy fails.

[ top ]

Always Reproduce

[ top ]

Lazy Player

The lazy strategies are divided into two subtypes: Simple Lazy and Scatter Lazy.

3A. Simple Lazy (Couch Potato)

[ top ]

3B. Scatter Lazy ("Coconuts")

Similar to the coconut tree, we found that the lazy strategy suffers from the problem whereby the children are all clustered near the parent. In the opening phase of the game, when food is much more abundant in locations farther from where automata started life, it's important to cover as much territory as possible. Thus we imposed this rule.

In a head-to-head matchup of Couch Potato and Coconuts (with scatter distance n = 4), neither one is clearly better than the other. In 10 trial runs, Couch Potato won 6 times and Coconuts 4. It appears that luck in finding food in the initial stages has a much more significant impact on the game than whether or not scattering is used.

[ top ]

Checkerboard

Our implementation of this approach suffered from some problems. The guiding rule of the implementation is that each automaton is always born on an even square, and he uses that as his frame of reference. When he moves onto an odd square, if he has sufficient energy, he reproduces on to each adjacent even square, in order to expand the checkerboard.

The problem occurs when two automata have both left their even squares and moved onto odd squares that border one anothers' even squares. They each think that there is a hole in the checkerboard in the place where the other was, and each reproduces to fill the even square "owned" by the other. Thus the even and odd squares now have automata on them, and no food can grow. Unfortunately, this happens too often to make our checkerboard implementation competitive in a multiplayer environment. This problem might be solved by better communication and better rules about when to reproduce.

[ top ]

Dumb Army

The purpose of the dumb army strategy was the spread the organisms out in a horizontal line, and then begin to march in procession up the board. If an organism was unable to reproduce to the east or west, but could reproduce to the north, the organism would reproduce to the north. This emphasis on reproduction led to an early swarm that dominated the board. There was a semi-conservative strategy built-in to temper the agressive reproduction, in that the organism would check to see if it had twice V, and that the area they were reproducing into containted food. In a case where the player could not reproduce, it would move north. In all other cases, it would stay put. In its implementation, this strategy had less to do with marching than the aforementioned ability to swarm the board at an early stage. The organism even seemed to dominate other organisms in early trial runs, however it was revealed that ultimately, the organism would die out almost as quickly as it built itself up, having focused on reproducing and not spending any time searching for food. So while this strategy may have been good in a destructive sense, to weed out other competitors, it had no staying power and was therefore abandoned.

[ top ]

Farm

farm  
   n.
    1. A tract of land cultivated for the purpose of agricultural production.

Farming is the use of organisms to block off a section of the board from other players, in order to allow the food in that area to grow unimpeded by the risk of organisms eating or resting on the area and preventing the food from staying at a high enough level. In general, this is done by having at least four organisms occupying the tiles around the food, forming a shape that blocks off the squares of the food. The walls have their energy replenished either by stepping in to the farm to regain their energy, or by having a queen inside the farm reproduce outwards in order to regenerate weakened areas of the farm.

The goal of farming is to maintain levels of food on the board in the steady-state phase that are similar to those at the end of the growth phase, to allow our team to maintain high automata populations.

Diamond Farm

The diamond farm is a farm where automata initially arrange themselves in a diamond pattern, and then try to expand the diamond, as energy permits, to enclose more and more area inside the farm.

An automaton following the diamond farm strategy follows these simple rules:

1. If the farm is not yet formed, generate more automata to form a minimal farm, filling the final part of the farm wall my moving rather than reproducing.
2. If an automaton on the wall of the farm needs food, it goes back to the farm center to get food.
3. If an automaton on the wall of the has more energy than it needs, it reproduces at right angles to its path to the farm center.
4. If both an automaton's shoulders are guarded by farmers, it walks outward, away from the farm center.

Under desirable board conditions, these rules generate fairly large farms. Furthermore, the farms are robust about expanding around obstacles and still preventing compromise of the farm -- automata will never expand if it would form a gap in the farm wall.

There are several problems with this general strategy that prevent us from using it in multiplayer games. For one, when an automaton on the wall must move back into the farm to feed itself, it may open a gap in the wall. With a large number of farmers, there is often contention in the farm's early stages for the square with the food supply on it, and gridlock ensues in the farm center. More simply, often the initial food supply is insufficient to sustain a large number of farmers. These problems are solvable, but require extensive communication and board analysis, so we gave up on the idea of large diamond farms.

A diamond farm immediately after it has been established.Some of the wall cells have sufficient energy, so they reproduce at right angles to the path to the farm center.	The topmost automaton sees that his shoulders are guarded and steps away from the center, expanding the farm.	The diamond farm has reached size 2, but now the lower-right automaton is hungry and is heading back for food, thus leaving a hole in the farm wall.

[ top ]

Queen Farm

Another method of farming which we implemented, but did not submit for the tournament, is Group1Farmer, which forms a farm with a queen in the center that regenerates the walls of the farms. The farm is formed when an organism finds a large pile of food in an area of the board that is relatively uninhabited. It replicates six times in order to form the walls of a 2X1 farm.

The queen replicates, forming the walls of the farm. The farm has been formed.

The queen then checks that the farm is complete and notes the energy levels of all the walls (constantly updated by having the walls communicate the levels to the queen, since simply measuring the levels at the time of birth might fail if the wall happened to end up forming its section of the wall on a tile that happened to already have food.) The queen then goes through a loop of three stages. It waits in the farm tile not containing food for a while in order to allow the food to grow. It then feeds itself for a while, although being careful not to deminish the food supply too much that it jepordizes the food supply of the farm. It then chooses a piece of the wall and moves towards it ordering it to step away from the farm in any direction that was free - where it will stay until it dies. If the wall cannot move in any of those directions, it stays where it is. The queen waits several turns to see if the wall will move, and if it does, it replaces that wall by reproducing into that tile, forming a new organism with more energy than it previously had. This regular pushing out the wall helps prevents the likelihood of two sections of the wall collapsing at the same time, which would cause problems for the queen that was trying to fill in the gaps. However, frequently, parts of the walls will die, and so the queen detects, using the information it maintains and updates of the relative energy levels of the walls, to detect when a section of the wall is close to dying. When the queen does so, it drops what it's doing, grabs some food to eat, and rushes to the tile next to the dying wall. After the wall collapses, it reproduces into the tile, filling the gap.

The queen asks a piece of the wall to move away. The queen reproduces into the empty slot left by the recently departed organism.

This farm is hard to establish -making it good as a supplemental strategy enacted whenever an organism goes onto a tile where the conditions are right, but not as a primary strategy. In order set up, an organism has to be on either a large food supply - or there has to be a very high q value - capable of maintaining all the walls in the farm. It also requires an empty area of 3x4 in order to establish. This prevents it from being useful in many cases, especially crowded boards or one in which food does not grow high or fast enough. But in the cases where it does form, the farm is self sufficent and does not require food outside the farm. This makes the farm survive under harsh p conditions, since it does not require new food to pop up, only growth of it's own farm, and it makes the player's population stabilize. This farm is very secure - because there are very few instances when the wall is not complete, and never for more than a single turn, since the queen has prepared to reprduce into that wall. Indeed, the walls to the west and north are secure enough that there is no way to infiltrate it. The walls to the east and south are more vulnerable, and another organism can infiltrate it if it happens to step or reproduce into the tile just as the wall tile died or moved out of the way - this tends to happen - albeit very slowly - when the farm is bordering another organism that is going through a very sharp population growth (often caused by the excess of food around the farm that is not utilized by the farm since it's not looking outwards). Also, given that parts of the wall move away from the farm, occupying spaces next to the wall, until these organisms die, they give an added layer of protection to the farm by prevening any opponent organism from filling up the spots from which they could infiltrate the farm. The farm tself is moderately complex - while communication isn't a big worry, given that the farm itself is secure from spoofing, coordinating the regenerating of the wall while making sure to maintain the food supply may be too much work given that this strategy is only useful in a few harsh conditions, as opposed to other strategies that have more general success with less involved organisms. One modification to this farm is to create multiple farms by having organisms that have left the wall try to create new farms. So I made a modification that had the north most and west most former walls search off and try to create their own farms. This allowed more farms to grow (although not always in a uniform pattern) and gave the overall player less vulnerable since all his eggs weren't in one basket, with an opponent now having to try to destroy multiple farms. Here is a screenshot of such a player in a single player game where it has created several farms (as well as a number of other organisms trying to form farms but cannot find the free space and large food supplies to establish them.)

A late game snapshot of a player that has produced several farms ( Click to enlarge)

As you can see from the graphs, the population and food supply have stablized somewhat. Another modification which we worked on, albeit only in theory, was a growing queen ant farm. In this case, wall organisms wouldn't be pushed outwards - merely replaced when they died out. The eight organisms forming the four corners would reproduce into their corners if they had enough energy. Whenever any two corners on one side got filled, that side of the wall would communicate with each other and so long as there was no one in their way they would move forward, increasing either the length or width of the farm by one. This idea had several flaws - mostly complexity, as well as the likelihood that the farm would expand beyond its means. If the farm was too big, an organism may use up too much food to reach the food or the point of the wall to be filled. The queen would also have to deal with several parts of the wall dying within a small period of time. The idea of having multiple queens came up, but the complexity was too high. Also, expanding walls around other organisms was also a complex problem to tackle. In the end, we decided that this idea was too far out of scope.

[ top ]

Smart ("Hunters and Farmers")

Our final strategy, "Hunters and Farmers," incorporates the spiral, lazy, and farm strategies into one player, and adds two new components: extra reproduction and genetic propensity to wander. Automata divide their time between hunting (using the spiral and lazy strategies) and farming (using the diamond farm strategy). The important part of this strategy is deciding when to use each other strategy.

The smart strategy first evaluates whether the current cell is suitable for farming, and if so, it starts a farm there. If farming is possible, we form a diamond farm as above, but we restrict the size of the farm to 1x1. We experimentally determined that 20 units of food is about the minimum worth farming -- lower and chances are there won't be enough food in the early stage of the farm to support the 4 automata required to defend the farm.

The basic formation of the farm in the Smart Strategy.

Next, if we are sitting on food, we follow the simple lazy strategy as defined above.

If none of the preceding rules indicated what we should do, we decide whether to wander or stay put. To make this decision, we first look at the known board parameters -- u, v, and K -- to determine an initial wander propensity. The formula we use is:

[(v - s)]^(1/2) Pw = 1 - [-------] [ u ]

In other words, the desire to wander is inversely proportional to the square root of the ratio of the energy cost of moving (v - s) to the energy benefit of food (u). We take the square root to scale the proportion so that a medium value of the ratio (like .5) produces a particularly low propensity to wander, since the unsquared ratio does not represent an accurate measurement at these levels.

To this we add our inherent wander propensity, which is fixed for each organism, but genetically determined so that evolution of wander propensity is possible. Our initial automaton begins with wander propensity at +.3, and children are allowed any value between +.4 and -1.0, with each child varying from its parent by one of {+.2, +.1, 0, -.1, -.2}.

It turns out that wandering earlier in the game is good, because food is abundant and tends to exist in greater quantities farther away from initial positions. However, wandering on more mature boards is bad, because it is expensive and less likely to yield much payoff, since there are likely other automata wherever your automata would have wandered. (This rule holds especially true for high values of u where mature boards support a large density of automata.) Thus the behavior we observe is that automata should become less likely to wander later in the game. This behavior is difficult to install into the automaton brain because it is hard for any one automaton to discern what the overall board density is. Therefore we use genetic mutation to solve this problem.

We tried to implement some way for automata to sense the maturity of the board by looking at how many automata they have seen in the past 1, 3, or 5 turns (weighting more recent observations more heavily). Unfortunately, there was not a consistently significant difference between measurements in early stages of the game and measurements at mature stages of the game, so we do not use this measurement in our decisions.

The strength of the smart strategy is in its adaptability to a variety of circumstances. For boards where wandering is expensive, it usually sees that and conserves its energy. For boards where food is evenly distributed around the board (high p and low q/p ratio), it expands quickly to fill the board and occupy food before other players get to it.

Best of all is the smart player's performance on boards with an uneven distribution of food (high q/p ratio), because the farms we create enable us to sustain significantly higher populations than simple hunting. In these boards, our farming population protects a large store of food and provides a base number of automata that generally does not decrease. The extra energy gained from the farms is used to field an army of hunters that is much greater than the hunters that could be sustained without the farms. These excess hunters compete with other players' hunters for what little food is available on the board outside the farms. The overpopulation of hunters leads to a famine that afflicts all hunters, so all hunters -- of all teams -- are continually dying off. However, our supply of new hunters stays constant thanks to the farms, while other players' hunters cannot compete and necessarily go extinct.

Hunters and Farmers player vs. all final submitted players at moment of victory, {x,y,p,q,v,u,K}={50,50,.0003,.02,4,20,53}	Hunters and Farmers player alone. Note that the overpopulation phase does not exhibit the usual sharp decline in the amount of available food, because there are enough farms to maintain high levels of food {x,y,p,q,v,u,K}={50,50,.0008,.02,4,20,100}
blue graph = number of automata, green graph = amount of available food Note that the amount of available food is maintained at abnormally high levels thanks to the farms, with the example on the right being more significant because of the greater abundance of food. Note also the sinusoidal shape on the falling edge of the green graph at the right as the amount of available food reaches steady state.

[ top ]

Tuning

To tune our player, we ran many tests, and searched for correlations between observed board variables and desirable behavior. Hardcoded tuning is most desirable, because if the player does the right thing from the beginning, it is much stronger than if it needs to adapt to the board first while other players are gaining strength.

However, we added genetic tuning of our wandering variable. This is the one variable we felt is most important and hardest to tune in advance, since the ideal wandering behavior varies based on the different stages of the game.

[ top ]

Tournament Results

In single player games, our organism consistently survives so long as P is greater than .00023. Below this point, we only survive so long as we happen to be lucky in the start game and happen to find food quickly.

In multiplayer games, we've found that our survivability tends to depend on U and V. In games where V is 4 and U is 20 (or where V is 10 and U is 20), we consistently survive when P is between .01 and .001, almost always being the sole survivors (or at least the team with the most energy). Below these levels, from around .001 to .0002, we are generally competitive, especially when we manage to reach food in time to survive past the initial phase of the game.

For multiplayer games where V is 18 and U is 20, we frequently walk ourselves to death trying to find food, although when P is high (eg, .008), we become more competitive, generally becoming extinct towards the very end of the game. For very low values of P, we tend to die out before we have a chance to start our farms.

For games in which V is 10 and U is 100, we sometimes survive - although not as well, since everyone performs better under these conditions. We tend to be more competitive when P is lower (but survivable), which allows our organism to set up and maintain farms, giving our organism an edge.

For games in which there is little food blowing in, we tend to die off because we sometimes wander around.

Farming tends to play a role when P is signigicantly lower then Q, such as when P is .005 and Q is .02, or when walking is very expensive. As P drops, farms are created more often and so long as q is high enough for them to survive, they play a more and more important role.

Our player does particularly well against other players in cases where p is 0.01, and u is greater than v. In particular, as the results below indicate, we dominate the multiplayer situations where u is double that of v. In these cases, where u and v are relatively close, the board will tend to be less dense, and therefore, wandering does well to find food and is conducive to the long term survival of our organism. However, we are not optimized for a more extreme situation, where u is 5 times that of v, and only come in first about half of the time in that situation. We don't do as well in this situation because we assume that wandering is better than it actually is under a situation where u is significantly greater than v. So since, in this case, the board will tend to be more densely populated, wandering is not a particularly appropriate action, since there is a higher probability that any available food we may find while wandering will already be taken by another organism.

[ top ]

Conclusions

Overall, we feel our player performed quite well in all of the tournaments where frequent movement was feasible. Our automata bring to bear a variety of rules and strategies that make it outperform most other organisms when food is scarce.

Better tuning for a wider variety of board circumstances would have made it more competitive in the situations where it did less well. By improving communications and working out some of the problems with larger farming and analyzing the board to determine how much to wander, our player could be developed into an extremely strong player.

[ top ]

Acknowledgements

• Group 3's player. We ran their player with debugging output on, from which we learned the importance of genetically varying propensity to move. We subsequently implemented our genetic propensity to move in a very similar way.
• Information on the Gambler's Ruin problem from http://mathworld.wolfram.com/GamblersRuin.html

[ top ]