Group 8 - Let's get together

Robert Tobkes
Akash Garg
James Rishe

Introduction

This project was particularly interesting because we had to develop a strategy that had to collaborate with the strategies of other groups. The challenge was using the information trails left on the game board by other players in a useful manner and developing a robust strategy that does not assume the behavior of other groups and finally can converge in finding others on the board. Questions that needed to be answered were how much to move, how to exchange information with a Friend in a neighboring square, and how best to find other players.

Throughout the development of the project, we considered many strategies, some of which did work and some of which did not. The rest of this document outlines the strategies we considered and implemented in our final implementation.

Group3Player1

Sharing Information - Meeting up With Friends

Our initial design included finding other players by relying on information that resides on the board. The base case was that we should always move to a square that is not empty and that square contains information that we do not currently have. This allows two Friends to meet when they are next to each other. The problems we had to consider with this case are the issues of deadlocking when neither player moves because it is waiting for the other to move, or swapping, when both players move simultaneously and swap places without information gain.

In order to rectify this case, we considered many techniques. One was that we create a convention that each direction of movement is associated with a number between 1 through 9. When we encounter another Friend on an adjacent square, the current player stays put if its direction number is greater than that of the adjacent player. This ensures that only one player moves and the two can share their information. This solution has a drawback however, since it does not scale well to strategies of other groups since they may or may not follow the same convention.

Another strategy uses probabilistic moving. The basic ideas is that when a player encounters another player on an adjacent square and the other player contains information that can lead to information gain, with some probability, the player moves to that square. This however does not always guarantee that the player will meet the other player. Hence, we do this step repeatedly until we gain the information we need. Another caveat with this approach is that if the other player moves away from the current player, then the current player is no longer adjacent to the other player and information that could have been gained cannot now. Another caveat would be to follow the player if it moves away from us in the next turn. We could also limit the number of moves that we have moved to chase the other player so that if the other player does not want to meet with us, we just find someone else who does.

Our final implementation used the aforementioned number scheme. When a player is on neighboring squares, our player will move to PlayerB if our player's number is less than that of Player B. The numbers are relative to to position of both players and follow the conventions set in IFCConstants.java. Our strategy is basically an 'L' on its side. If a Friend appears in the section labeled "Stay Still" then our player will not move. If a Friend appears in the section labeled "Move" our player will move to said location.

However, this strategy is unsuccessful if another player is coded exactly opposite of our player; this other players stays still when we do, and moves when we do. In order to prevent such deadlocks, our player will timeout after 5 turns, and ignore the adjacent Friend. Furthermore, this strategy is not succesful on a filled board; therefore, we chose a different strategy in such a case. Instead, our player will move randomly if the board is more than 75% full. This threshold was determined with offline testing of various levels and then analyzing the results. (In fact, random movements performaned as well as our player when the board was more than 60% full. However, considering the benefit or all players, and the difficulties caused by complete random movement, we instead chose the higher threshold.)

Board Information

Another branch of strategies we looked into was developing strategies to find other players while making the most of the information presented on the board, board coverage strategies. During the initial stages of our implementation we considered following a greedy approach and following a trail left by other players on the board. The basic strategy simply goes to the next adjacent square that has the most amount of needed information. If there is a tie between squares, then a random move is chosen.

The problem with blindly following a trail that contains the most information is that during beginning stages of the game there is not much information on the board so most of the moves made are random, while during later stages of the game the board becomes "dirty" and the information on the board does not clearly indicate the positions of other players. Since the trails left by players are permanent, a square with a particular player's information only indicates that the player has visited that square, but does not indicate when. Thus, the player could be very far from that square or very close to it. Thus simple chasing algorithms do not work. Our first modification of chasing was to have our player move to chase, but to move to squares that had been visited the most. This idea failed, and was dropped, because players too easily became caught in tracks and did not successfully cover the board. Also, simple chasing does not work due to the cyclic paths laid down by the horizontal-vertical players. In order to account for this, we will stop chasing if we have not found our target in the maximum of the board length/width turns.

In order to fill the drawbacks of simple chasing, the greedy approach was later modified to include a weight function for each adjacent square that will determine where to move next. The weight function initially computed for each adjacent square the amount of information on that square that we do not contain, as opposed to the total information on that square. Each square is also weighted less for each amount of information unit. The basic premise being that the less total information on a square then the fresher that information is going to be, making it suitable for chasing another player. Finally, a square that contains the information of the current player, is weighted a little less. This makes sure that we do not backtrace to squares that we have visited already and have more board coverage. Below is a the weight function for a square:

W(x) = -1*numPlayersTraveled(x) + -1*ifCurrPlayerVisited(x)

Lastly, in order to break ties, a random value less than 1 but greater than or equal to 0 is added to W(x).

The weighted function described above does better when the board is large and fresh. However, when the board gets considerably "dirty" it suffers from the same shortcomings as the greedy approach described above. As a result we decided that board information in trails is will not make up the best solution because trails are permanent and do not have time associated with them. If the trails decayed after a certain amount of time, then one could infer at least to a certain extent the approximation of the location of another player and how far they could be from that spot.

Bees and Flowers

Another strategy we considered was the concept of Flowers and Bees. This strategy tries to emulate nature. Some players act as bees and some act as flowers. The bees are rampant and are constantly moving to find other players. However, the flowers basically stay put in one place and wait to be found. The basic premise is that a right combination of bees and flowers can result in an optimal strategy. The flowers can act as retainers of information and can pass that information to other players that come by them. This strategy also works well in multiplayer games because it does not rely on any assumptions made on the techniques used by other players. Of course even this strategy has its downfalll when there is an imbalance of bees and flowers. A good mix for a particular size board can be difficult to find. A simple method for determining bees and flowers is to simply assign states to the players at the beginning of the game with some probability. This method may not work in single player mode gameplay since there is a chance that all of the players will end up being a flower and no one will move and the game never converges. Consequently, this can be useful in multiplayer games as it might be safe to assume that other teams will have players and that are explorers. On the other hand, we do not want to have all bees on the board. Deadlock situations like the one listed above when everyone is a flower can be modified by dynamically changing the strategy of a given player. Some groups have considered changing a flower to a bee and vice-versa when a player has not encountered another in a very long time. Determining the optimal timespan to wait can be difficult however and should be a function of the board size.

We considered a variant of this stategy and did not make a clear distinction between bees and flowers. Instead we associated each player with a speed attribute. The attribute determines how long the player waits between turns before moving. So a speed of zero will result in the player moving on every turn, while a speed of three will result in the player waiting for three turns in between moves. This decrease in movement frequency can be used to give the same advantages of the flowers and bees strategy. Our initial design considered all players at the beginning of the game with a speed of zero. The players then move according to some movement strategy like the greedy approach to find other players. If the player has not seen anyone for L x W number of turns (where L is the length of the board and W is the width), then their speed is increased by one. This effectively slows down players and slowly makes them flowers if they have not seen anyone in a very long time. A shortcoming of this approach however, was discovered in class when our player was put against "DumbPlayer". The problem is that when players are put into a configuration like the Figure shown below.

When players are spread out convergence may not happen for a long time. Thus, our players will decrease their movement frequency and everyone becomes very stagnant. This is especially noticable when playing with "DumbPlayer". DumbPlayer does not converge very quickly since every move is a random one, and this results in very slow movement rates since our players do not find anyone else. This adverse effect is even more noticable in large boards. Therefore in order to rectify this downfall, we increase the length of time we do not see anyone before increasing our speed every time our speed has increased. For example: if our speed first increased after not seeing anyone for 100 rounds, then we would increase our speed attribute if we do not see anyone for another 150 rounds instead of 100 rounds. This helped in larger boards and against dumb player but still has shortcomings. After some time, the threshold will grow to high and the player will only be a flower after a very long period of not seeing another Friend. We deemed this to be too ineffective. Instead we set a max speed of 10 to avoid the above situations when all players are flowers. Lastly, we also added a probabilistic approach to this strategy specifically banning 20% of our players from developing into flowers. This ensures that not everyone on the board will be slowing down if they have not seen each other in a long time.

Board Coverage

One of the main challenges faced was what movement strategy to choose when trying to find other players in different parts of the board. In a fairly small scale board like 25x25, board coverage is not of utmost importannce, however, as the board size increases it is important to cover as much of the board as possible in order to find other players. There are several strategies to accomplish this.

Diagonal Movement

This is a very simple strategy in which the player moves across board diagonally. Moving diagonally is better than moving straight in traditional East, West, North and South directions because each movement in the diagonal direction provides a little bit more coverage of the surrounding board. The basic diagonal movement can be shown in the figure below.

This approach does need some careful considerations. If we simply set a rule that does not deviate from the diagonal path that is originally laid out we can run into the problem where not every square on the board can be visisted. A simple rule to change diagonal's we are travelling on will alleviate this problem. Spiral Movement One basic strategy is using spiral movement for board coverage. Each player starts in square and sprials around a particular area throughout the board. The basic approach is illustrated in the figure below.

This approach can also be modified so that spiral movement occurs only in local regions and if we have not seen anyone locally by doing the spiral, the player moves in a straight line for X number of turns and starts the spiral again in a different area of the board. The number of turns to move away from the previous sprialling area is dependent on the board size. The spiral movement has good properties in that it uses a form of diagonal movement but covers local regions in the board. This can be useful in finding other players that are residing in a particular area. Integrating the sprial movement with the bees and flowers strategy had some adverse effects, since the spiral only covers local regions, in large sized boards, players slow down and do not find other players since they do not cover a large aspect of the board quickly. Sprialling works very well when exploring local regions on a board.

Zig-Zag Movement

Another variant to the diagonal movement approach is to move in a zig-zag formation. The basic movement approach is illustrated in the figure below.

The zig-zag movement however suffers from exploring areas in two directions since it does not travel all the way across the board. In the figure above the shaded region is left unexplored for a long time. Thus when the pink player above actually gets to where the blue player is, the blue player might have moved to where the pink player is originally. This movement pattern can lead to long convergence times.

Our Approach

We decided upon a simple, focused wandering approach where our players are split up into 2 groups, with equal probability -- An "up" group and a "down" group. The "up" players are only NOT allowed to move South, Southwest, or Southeast while the "down" players cannot move North, Northwest, or Northeast. Of the allowed directions, one is chosen randonly as long as it is not to the square that we have just come from. For example, our player can never wander east one turn 1 and then west on turn 2. We decided on this focused wandering strategy because we liked the liberal movements achieved by random wandering. We observed that strict adherence to one of the above board coverage strategies to be too constricted and detrimental to the overall performance of our player.

Our Final Approach

A combination of the all of the above approaches was used in our final player. Our player's movements are driven by smart chasing (move to the square with the most new information, use a weight functions if there is a tie), when a trail is evident. When no trail is evident our moves are dictated by the wandering with a purpose strategy. Finally, most of our players can become flowers when they have not met up with another player in some time. Also, we have coded strategies for full boards (over 75% - move randomly), and large boards. In a large board (greater than 45 x 45), our players will not become flowers if we only need 20% more of the information to finish the game. We added this because we observed that the reason the game was not finishing was because there were flowers in remote parts of the board. So, instead of waiting for someone else to find these few remaining bits of information, we try to actively seek them out. The threshold was once again chosen with offline testing. In our testing sessions, done with 9 players, much of the game was spent trying to get the final 1 or two pieces of information, yielding 11% or 22%. We used the a number much closer to the latter because we decided being more active was better in this scenario.

Tournament Results and Analysis

Our single player performance is as expected. Because we did not implement a strategy that is optimized for single player mode (such as the horizontal-vertical player or the board-sweeper player), we have not excelled in this mode. However, out of the 26 single player games run, our player finished in the top 50% of the class, 46% of the time, which we believe to be qutie respectable.

In the multiplayer tournament, our player did not perform well in the large board, many player case. We believe our player must have become a flower and none of the other players on the board could find it. Furthermore, in the games in which we finished in place 10 or worse, we did finish within one standard deviation from the average in 75% of those games.

Group8Player3: Pinkeye

The Idea

The initial idea was to keep a memory of all cells seen, the round in which they were seen and which traces were found there. We would have used this information to go to the place where we had most recently seen the trace of a piece of information that we wanted, and that would be the starting point for a search for information. We decided before we started that by the time the player got to a place that it was chasing based on memory, the player with that information would probably be long gone. The idea of keeping a history of the times when certain squares were seen, however, stuck.

The Strategy

Pinkeye starts off taking three steps at a time in random directions. Other walk strategies were attempted, but the random one seemed to work the best. It seemed that other patterns often had the players walking parallel to each other, possibly due to troubles with Java's randomization, discussed later.

After every step, the player checks for the trace of a piece of information that it does not have yet and if it sees one it will begin following that trace until it discovers another player. It chooses its next step by choosing the spot with the right trace that was seen the longest time ago, in order to be always moving towards fresh information. Once the player sees any other player it will begin interaction, described below.

Interaction

We thought of a plan for resolving motion discrepancies among our own players during the first class and proposed it in the second class. Essentially Pinkeye will sit still if it sees nobody above or in the three directions to the left, but does see somebody below or in the three directions to the right. If it sees anybody above or to the left, it will move towards it, favoring the directions in this order: NW, N, W, SW. In this way, a clump of Pinkeye players will gradually but quickly make their way to a single square in the top-left of the clump without deadlock, as long as a wraparound of the board does not occur (discussed below).

This scheme empirically meshes well with other teams' strategies, but obviously in some cases it would not. The process of waiting and moving towards other players is given the term "interacting," and once a player has interacted for four turns, it goes into "don't bother me" mode for four turne. When in this mode, the player does no interactions with other players, but reverts to the behavior of randomly searching and then following paths. This provides the following benefits: A player stuck waiting for another one that does not behave in a compatible way will break free and set off on its own, perhaps to communicate well with the other player later. A player endlessly chasing another will also break free. In the event of a wraparound, the randomization of the searching will mix up the players and cause them to rearrange themselves. Finally, players that have succesfully finished communicating are given four rounds to spread out from each other and not fall back into more interacion that would provide no new exchange of information.

Problems

Randomization

We found through experimentation that since the players are created very closely in time, a lot of them seem to receive the same seed for their random number generator and as a result, they can end up exhibiting identical patterns when they should be acting differently. After much frustration, we found that we can seed the random number generator with the method hashValue() that belongs to Object and is seemingly unique for the different players, and once we did that, the random numbers were a lot more useful.

Wraparound

We tested this player early in the full board case, and found that it endlessly followed itself, partly due to the poor randomization. After fixing the randomization problem, the player did work, and quickly, on the full board. It generally ends up chasing the others for 4 turns, followed by a "shuffle" and then more interaction. We implemented a "really don't bother me" mode, in which the player ignores all stimuli and moves randomly for the first 4 rounds of the game if more than 80% of the board is filled with players, and then goes into normal function. These first four rounds are the only part of its behavior that deviate from its usual behavior, but they do improve the results a bit.

Endless chase

This player attempts to follow paths that it believes will lead it towards another player. The drawback to this approach is that it could be following a player that is continually running away, or it and another could be following each other's paths around the board infinitely. To ramify this, we implemented a record of moves taken while tracking, and if the player finds a dead end or it runs back into a part that it has seen recently, it will backtrack until it finds a new path to follow for the same piece of information. Considerations were also made to recognize a situation of endlessly circling the board. Unfortunately, these two improvements were finished a few hours after the submission deadline and were not included in the final player.

Tournament results and analysis

Top 3 in multiplayer (without dumbplayer): 40%

Top 3 in singleplayer (without dumbplayer): 52% 

Top 4 in singleplayer (without dumbplayer): 81%

Pinkeye performed surprisingly well. The general trend is that it did better in competitions with more players, and this trend is especially strong in the multiplayer tournaments. This deficiency is probably due to a failure of the tracking algorithm in finding the other players in a sparse board, perhaps getting lost and turning back away from the other player(s). The improved pathfinding code that did not make it into the tournament maybe could have helped, or it may have not, but overall, Pinkeye was a success.

Work