Project 1: Rectangles

Group 2: Eric Li, Ludvig Ungewitter

Introduction

The first project, Rectangles, was a game played on a grid board where each player controls n robots and tries to encircle the maximum amount of space on the board by constructing rectangles whose border consists completely of that player's color. The game ends when an agreed on number of moves has been played without any new cells have been claimed. The goal of each group is to implement algorithms that govern the behavior of the player's robots in order to be the best at winning the game.

Strategies

The dilemma with the game is that there is no general optimal strategy. Different strategies will apply to different board densities and to different players. Also, the opponents are unknown prior to the start of a match. The option of choosing a strategy that fits a certain opponent, therefore, does not exist. However, you do have access to various parameters such as board size, number of teams, number of robots, and number of free spaces. These parameters can wisely be used as guidelines for what strategy to use at what time.

Choosing the correct strategy based on parameters is no trivial task. There are many different directions that can be taken. Each robot can be constructive or destructive. A constructive robot tries to build rectangles, whereas a destructive robot tries to destroy other players' rectangles. A team can assign all its robots to be destructive or just a few. Different constructive techniques have their own tradeoffs. For example, a constructive player may attempt to enclose a large rectangle. If successful, this will net the player a large number of points relatively fast. However, the larger the rectangle the longer it takes to complete and the higher the risk of having another player interfere. If the player attempts to construct only small rectangles, there is much less risk of being disrupted but the points earned is also much less.

From analysis and discussion of the previous year's players, we were able to see the various strategies that the previous groups had come up with and their relative effectiveness against each other. Since we had access to the code, our group was able to do some extensive analysis of our own to profile the various strategies we had seen and use that to innovate our own strategy. In general, the primary strategies can be categorized into either offensive or defensive types, so we shall give an enumeration of the important ones (We leave out strategies such as walking around the border or moving randomly or being idle because we don't consider those viable strategies) and what we noted about each. The reason we go into quite a bit of detail even on strategies we didn't use in our final player is that we have actually implemented and tried our own variations of all of these strategies so they were important stepping stones for us.

Offensive Strategies

• Army

  The army strategy consists of having all the robots assembled next to each other in a straight line and moving forward in a path perpendicular to that line so as to sweep out area as the formation moves along. The strength of this player is the immediacy with which points are scored. This minimal delay between start and point scoring can be a decisive factor when up against players like the chaser. Another advantage of the army player is that in late game scenarios where there are various pockets of white spaces left, it can be pretty effective in collecting those squares by simply sweeping over them. In comparison to a search and fill strategy that is much more complex in terms of logic and implementation and may not be as efficient in the end. The weakness of the army player is that it needs a large number of robots relative to the board size in order to be truly effective. This is because with the same number of moves, robots acting independently and drawing out large rectangles can gain area much faster than if they were to move in the army formation. The robots between the first and last ones in the army formation essentially are wasting their moves in order to claim the area the army has walked over without delay.

  

• Cavalry

  The cavalry player lines up his robots on a line, also, but allows there to be a fixed amount of space between any two of his robots. When the robots start to move, they work in pairs. Each pair draws out one half of the rectangle that the two of them are responsible for filling. Once they're done, they continue with the same behavior on more white space. The cavalry player is similar to an army player that has traded in the immediacy of claiming squares for efficiency in being able to claim bigger rectangles in the same number of moves. Thus, the two players have similar strengths and weaknesses. The advantage of the cavalry player is that it can be tuned by adjusting the space between the robots and the dimensions of the rectangles they try to complete. In this way, it can reduce the severity of the army player's weaknesses.

  

• Cooperation

  Cooperation is a fuzzier strategy than the others in that there are numerous different types of robot logic that can exhibit the same behavior. Essentially, cooperation involves controlling several robots in the vicinity and telling them to work together to complete a rectangle. Other peripheral logic such as filling in only those squares that aren't already filled in or redoing what the enemy disrupted can be added to the mix to improve efficiency. The strength of cooperation is that if done right, it can be a very intelligent strategy with virtually no weaknesses. It's logic closely resembles how a human being would approach the game. Its weakness is the complexity of implementation, which due to the principle of emergent properties can be quite difficult to manage. A strategy like cooperation must take many different factors into consideration as well as being able to manipulate and interpret those factors in a way as to produce robot behavior that is most efficient. Another weakness of cooperation is that it could potentially be very ineffective against disruptive players because as the professor mentioned in class, it only takes one uncolored block to stop the completion of a rectangle making destruction potentially a lot easier to achieve. For example, if one enemy robot decides to sit on the border of a rectangle that the cooperators were trying to complete, that rectangle must now be abandoned.

  

• Grid

  The grid strategy places robots evenly spaced on adjacent edges of the board and has them travel in straight lines across the space of the board. This way, grids are drawn out by the robots as they move. Ignoring disruption, this strategy has one of the highest rectangle fill rate efficiencies with respect to number of moves. It's able to achieve this because every robot is essentially drawing borders for some rectangles as its moving. Almost no moves are wasted. At the same time, the delay to claiming the squares can be tolerable if there are enough robots on the board. Like army and cavalry, this player relies on having a lot of robots to be truly effective.

  

• Pattern

  This is a broad category we have included to describe any players that have their robots move in certain noticeable patterns. This includes the players that move in snake-like patterns up and down the board or diagonal squares from one corner to another. No matter the pattern used, the strengths and weaknesses are essentially the same. Depending on robot density and some tunable parameters the pattern player will have different efficiencies. In essence, the army, cavalry, and grid players also belong in this category. But they were used extensively and are special cases so we decided to go into a little more detail about them. The quality that characterizes all pattern strategies is that they are not reactive strategies. They have their robots follow certain predetermined patterns without consideration for what the other players are doing. This severely limits their effectiveness.

  

• Search and Fill

  This strategy is somewhat similar to the cooperation strategy in that it is relatively complex with respect to pattern type strategies. Also, it and cooperation are really reactive type strategies that can adapt to the changing situation on the board. Of course, you could add some reactive logic to any pattern player and say that they also have this kind of behavior, then you're really using a mix of a pattern and a search and fill or cooperative strategy rather than a pure pattern strategy. The slight difference between the cooperative player and the search and fill player is that the cooperative player focuses on having several robots work together while the search and fill player focuses on finding white space to claim whether it's using one or multiple robots. The search and fill player has similar strengths and weaknesses to the cooperative player. In fact, both incorporate each other and are usually used together in the same player.

  

These are the primary categories of offensive strategies. There were numerous variations on each and some groups used more than one. But they could all be broken down into these.

Defensive Strategies

• Anti-Chaser

  This was a popular defensive strategy this year due to the effectiveness of the chaser being demonstrated early in the progress of the project. It's aimed specifically at fighting chasers. There are many ways to combat chasers with varying degrees of success, so this category has many sub-categories. One strategy was to have a robot that follows the chaser in order to undo the damage that done by the chasing robot. Another strategy was to stop completely or to chase the chaser so that neither player makes any further progress. From our observations, we realized that once the chaser has caught up with the robots of a particular player, that player can no longer make any significant progress. The key then lies in using the amount of time before a chaser catches your robots to capture the most number of squares. Once your player has the upperhand in scores, then you can simply use a counter strategy to make sure the chaser doesn't gain any more points and the game will end with you being the winner. This was indeed the case in the latter phases of the project when quite a few teams including ourselves spent a lot of time optimizing our player to claim squares quickly in the early game.

  

• Chaser

  This is a simple strategy that has each of the chaser's robots target one of the opponent's robots and tail them exactly. This is an extremely effective strategy in that by following the opponent perfectly, he no longer has the ability to add to his board realestate since every square he claims is immediately converted to a square of the chaser's color. This is an extremely devastating strategy in two player games, and virtually impossible to defeat if the initial placement of the chasers is well chosen so that the distance to any one of the opponent's robots is fairly small, thus, not giving them the time to claim much space before being locked down by chasers. The greatest weakness of the chaser is that it is utterly ineffective in multiplayer games. At best, the chaser can only nullify one player. If that player has no anti-chaser strategy, then the chaser's chances of winning depend completely on how well that player does in multiplayer scenarios. However, if that player employs anti-chasing, then both the chaser and that player are now ineffective leaving the rest of the players to win the game.

  

• Disruption

  Disruption is a general category that includes any strategy that tries to prevent other players from completing their rectangles. The advantage of disruption is that it takes less effort to destroy than to complete rectangles. However, against efficient offensive strategies that simultaneously tries to complete several rectangles, disruption can prove quite ineffective. For example, if the disruptor successfully prevents a smaller rectangle from being completed while the larger enclosing rectangle is made, then the work of the disruptor is wasted. Thus, effective disruptors must be highly intelligent and be able to figure out which rectangles are the ones that matter. Another weakness of the disruptor is that its virtually useless against other defensive strategies.

  

These are the primary defensive strategies that we knew of. There were other defensive strategies out there that we didn't talk about because we didn't explore in those directions. From these strategies, both offensive and defensive, we realized the key factors governing the effectiveness of a player and the tradeoffs involved in using them, and proceeded to innovate our own player.

Group2 Player

In actuality, no player is as simple as any one of the strategies detailed above. All players use combinations of several strategies and even switch strategies based on the game conditions. Our player is no different. Below, we explain the various strategies that we tried.

Primary Strategy

From the beginning, we decided that our primary strategy would be a chaser. Though the previous year had not produced any effective chaser players, we reasoned that the complete chaser is essentially the most effective two player strategy by analyzing its characteristics qualitatively. We successfully implemented a primitive chaser as our first deliverable, and in tests against last years players, we were able to beat all of them in all two player scenarios. Because of this, we decided to stay with the chaser as our main strategy and build on it.

First Chaser

The first incarnation of our chaser had each of our robots target each robot of the opposing player with the same index. This was essentially a random targeting. We only wanted to see how effective chasers are at first. Later, we developed much more complex targeting mechanisms. The chasing algorithm we used was simply to use the coordinates of the enemy robot and our own to calculate the x-distance and y-distance between the two, i.e. the horizontal and vertical components of the cartesian distance. By the sign of the distances, we could then tell the direction of the other robot relative to our robot and set our moving direction toward that robot. Despite its simple logic, it was effective against all of last year's players. Yet, there were obviously many improvements that could be made to this player.

Rectangular Path

One of the first inefficiencies about the chaser that we noticed was the way in which it headed toward the enemy. It tended to zigzag. One reason for this was that our robot always checked the north/south direction before the east/west direction, so that if the enemy is directly east of us and we're heading towards it, if it heads north one block, we will also head north one block and same for south. This way, instead of making a straight line towards the enemy, our robot draws a broken zigzagging line. To correct this, we made our robot always head in the same direction that it headed in the previous round as long as the distance between our robot and the enemy robot in that direction is not zero. As expected, our robots now headed in unbroken rectangular paths towards the enemy. This made the chasing paths more useful as edges of rectangles that often emerged from several chasers walking by each other.

Target Furthest

Once we find the opponent, our chasing path is no longer necessarily rectangular since it's then determined by the movements of the enemy robot. So we decided to chase the enemy robot that is furthest. That way, our robots draw out the longest stretches of rectangular lines on their way. We achieved this using a matrix of distances between our robots and the enemy's robots. Also, we placed our robots evenly spaced around the board so that when they started moving, they naturally formed grids as they headed toward the enemy robots. Our goal was to increase our ability to capture squares at the beginning of the game prior to reaching our enemies. Our improved chaser was highly efficient and extremely effective against most regular players. But against chasers, our targeting strategy actually made us weaker. See the section on targeting nearest for further explanation.

This is an example of our player using rectangular paths and target furthest together with the perimeter initial placement to achieve grid formation as each of our robots head towards the enemy's robots. Notice how the grids are emerging in the second screenshot. In the third screenshot, we see the devastating efficiency of our initial grid strategy.

Proximity Chasing

After the release of our player, various teams optimized against the chaser strategy. One strategy that existed even in last year's players was stopping when an opponent's robots are near. This is damaging to the chaser as the chaser relies on the enemy's movements to capture further area. Thus, to counter this, we added proximity chasing. Bascially, we keep a variable called proximity that determines how close we want to get to the enemy before we consider ourselves to have reached him. In our previous chasers, we always assumed a proximity of zero, i.e. we aimed to be exactly where the enemy was. With a proximity of two, our robot stops when we're within two squares of the enemy robot. Given the appropriate proximity, the enemy would not be able to know that it is being chased. Therefore, we effectively combat the situation where the enemy slows down or stops. This proved to be quite effective against most players. However, having a proximity also means the enemy actually has a few squares belonging to him at any given time that he is being chased. In some situations, though rare, this allowed him to complete rectangles.

Back Off

A problem with proximity chasing is that we don't know within what proximity the other players consider themselves to be chased. In order to accommodate all players, we could hardcode a large proximity, but this would greatly weaken the effectiveness of our chaser. So instead, we implemented automatic proximity tuning which we call back off. Basically, proximity starts at zero when we first chase. When we get to the enemy, if he stops moving for a certain number of rounds, we know that he has detected our chasing, so we increase the proximity variable by one and then retrace our steps and back away from the enemy by one square. If the enemy continues to stay still, we repeatedly increase our proximity and distance until the enemy starts moving again. At that time, we continue with our chasing making sure to stay at that distance so that he doesn't stop again. To guard against having too high of a proximity that we're no longer effective chasers, we set a maximum that the proximity can be. To avoid situations where the enemy stops for some other reason instead of being chased, we decrement the proximity variable every eight rounds because the closer we chase the more effective we are. When we tested back off, it had mixed results. Against some players, it did exactly what it was meant to. Against other players, it didn't have much of an effect in preventing freezes. We conjecture that that has to do with the way our enemies detected chasing, since back off assumes they do it by distance.

This is an example of our player using proximity chasing and back off. We intentionally used only 4 robots so the chaos of robot movements doesn't obscure the chasing trail. Notice how in the last picture, our robots have adjusted themselves to chase at the optimal distance so that the enemy doesn't notice our activity.

Anti-Freeze

In some cases, some players stop all their robots once they detect chasing. This creates absolute stagnance in the game and the winner is the player with the higher score at the time. If it's not us, then our chaser is in trouble. In the case that proximity and back off fail, we fall back on a simple anti-freeze strategy where if we detect that all enemy robots have stopped, we reacquire targets of chasing for all of our robots by finding the furthest enemy robot to chase. This way, our robots will start moving again even if the enemy's don't. The idea is that as we're moving, we'll create more rectangles along the way so while the enemy stays still, we can still get points.

Target Nearest

A problem we had with target furthest is that against other chasers, this strategy made us quite ineffective. In fact, our target furthest chaser consistently lost to our most primitive chaser. The reason for this is that a chaser becomes effective the moment it gets close enough to the enemy to nullify it while gaining territory for itself. In targeting furthest, while we're going after the enemy robots furthest away, there are already enemy robots chasing us and nullifying our paths. So the grid work that we lay out as we get to the enemy is consumed by the enemy and helps him make rectangles instead of us. Because of this, we decided to switch to a target nearest strategy. It proved much more effective against chasers. We often ended up in draws instead of losing. However, against group 7's player, we were still losing more often than we were winning even with target nearest. This is because there's a targeting strategy that gets to the enemy even faster than target nearest. And that is what we call touch chasing.

Touch Chasing

Touch chasing is the idea of chasing the first enemy robot that touches our robot. This idea can be generalized to chasing the first enemy robot that comes within some certain distance of our robot. This makes targeting immediate and decreases the amount of work needed to find the enemy robot since we let it come to us instead of going to it. One other advantage of touch chasing is that it lets our robots do something constructive before beginning to chase the opponents robots. If we use this time correctly, we can gain a large point advantage before nullifying the other player. The weakness of touch chasing also lies in this same idea of a constructive beginning. As long as the enemy doesn't approach us, we're essentially behaving like a regular offensive player. If this is prolonged, our player no longer leverages the strengths of being a chaser and the contest then becomes just a matter of who captures squares quicker between two offensive players. And since we're a touch chaser, we can't be too elaborate or ambitious with our initial offensive strategy since our robots will abandon their tasks as soon as someone is near making it quite a task to find the right starting strategy.

Score Checking + Absolute Chasing

One simple improvement to a chaser is to check the score before chasing. If we're winning, then we can do absolute zero distance chasing. No matter what the opponent does, the game will then end with us winning.

Army Start

One strategy we tried in conjunction with touch chasing is to start using the army strategy. The army strategy captures squares relatively fast without delay, and we wanted to use that to our advantage. This seemed like a good idea, and against some players it was. However, most good players optimized around having a quick start strategy, and the longer that player is allowed to roam free, the more damage it could do to us. Thus, the army strategy proved ineffective against the optimized players.

Cavalry Start

As an alternative to the army start, we used the cavalry start, which has a higher efficiency. Cavalry start faired much better. It was able to defeat most players, even some optimized ones. However, the cavalry start suffered from the weaknesses of the cavalry, i.e. it would move through squares regardless if they were black or not and that was a waste of moves. Also, cavalry depends more heavily on the formation of the players and their adherence to their paths, so when robots went off to chase other robots, the cavalry became ineffective.

Here, we have an example of our player using cavalry start and touch chasing. Notice how the cavalry formation is broken because some of our robots have detected enemy robots and went off to chase them.

Multiplayer Target Switch

In multiplayer games, the chaser was completely ineffective. To remedy this, we came up with the idea of having the chaser target the robots of whichever player is winning. This can be determined dynamically, so every x number of rounds, we check to see who is winning and go after that player. Theoretically, this strategy has the ability to pick out the best player among the rest and win by shadowing that player. In practice, due to the fact that almost all the good players have anti-chasing strategies, whenever our player targets any player, ourselves and that player nullify each other and are essentially taken out of the game.

Multiplayer Strategy Switch

In the end, due to time constraints, we couldn't explore all the strategies we had thought of so we ended up partitioning game conditions into several possibilities and then selecting one strategy we thought was effective in that scenario. For our two player games, we used the version of our chaser that did best overall against all the players. Then for medium robot density multiplayer games, we used group 5's cavalry. For games with a large number of robots on the screen, we used a search and fill strategy from group 8.

Initial Placement

One important component of our strategies was initial placement of robots. We focused on optimizing initial placement to best support our chaser strategy. For that, we tried placing robots evenly spaced around the perimeter of the board. This has the advantage of creating the most extensive grid when used in conjunction with target furthest. But since we switched to touch chasing, our initial placement then became a grid formation of four lines cutting through the board like a tic tac toe game board. The distance from each line to the edge of the board is exactly one fourth the length of one side of the board. This is to minimize the distance between any one of our robots and any enemy robot so that it is at most within a few squares of the quarter length of a side of the board. Used in conjunction with target nearest or touch chasing, this placement proved quite effective.

This is an example of the perimeter placement. The advantage of this placement lies in being able to create grids as our robots head towards the enemy's robots. An example of this tactic is shown in the pictures several paragraphs above.

This is an example of the grid placement. In this picture, the distance from each line to the board edges is actually 1/5 the length of a side of the board. In our final player, we stuck with 1/4.

Another initial placement we tested was to put all our robots around the perimeter of a square. Once the game starts, the first thing we do is to complete that square using the minimum number of moves before we head off to chase other players. If the square is large enough, we will gain a sizable advantage in points much sooner than the opponent, and once we initiate chasing, we would be the guaranteed victors. However, this placement turned out to not work so well because we can't know in advance where the opponents robots will be. Often times, the enemy robots appeared right next to or even on the border of our square making completion very difficult, especially if the opponent is also a chaser, in which case completion is impossible since they would never leave the border. In the end, we abandoned this placement, though it's very possible to develop this into an effective starting strategy by adding more logic to the way we complete our initial square.

This is an example of the square boundry placement. Notice the enemy robots right near the square's edges. The random appearance of those are what caused this placement to be ineffective.

Tournament Results Analysis

Our player, Group2PlayerF, consistently ranked second in the 2 player games with 10 robots. In the other 2 player games, with either more or less robots, our rankings fluctuated. The weaker performance in games with few robots can be attributed to the fact that the other players tried to quickly capture squares in the beginning. With a low robot to board size ratio, it took a while for our robots to get to the enemy robots, thus, giving them sufficient time to score some points. Once we caught up with them, they most likely stalled until the game ended. With a higher than average robot density on the board, the same phenomenon can occur if the enemy robots can leverage their closeness to quickly score some points before switching to an anti-chasing mode. Our games versus last year's players came out as expected. We won the majority of our games since we spent much of our time optimizing using the old players.

In multiplayer games, we performed as expected in the lower density games where our robots could take advantage of the cavalry strategy by occupying the abundance of white space quickly. In higher density games, the constant disruptions caused by all the chaos of the other players moving around made our player ineffective.

One factor we feel hindered our ability to perform better was not knowing the anti-chasing strategy used by our opponents. We tried to accommodate for most of them but were successful only to a degree. Nevertheless, our tournament results seem quite satisfactory.

Conclusion

Rectangles was a very interesting and entertaining first project to work on. One of the areas that would be interesting to delve further into would be to come up with a strategy that can consistently and effectively beat an optimized chaser in two player games without using chasing of its own, or prove that no such strategy exists for the given game conditions. One other area we wanted to develop further but couldn't due to time constraints was building a highly intelligent cooperative search and fill player. It was actually one of our initial ideas to use a statistical model to model the board and a hidden markov model for the states of our robots. But discovering that the chaser had such effectiveness without the onerous burden of complexity, we chose to go down that path instead for this project, as often times in software engineering, simplicity and project completion is more valuable than complexity and unfulfilled potential.