EEI - Proper Enemy Flanking and Herding, Part 1

Previously, I noted that there were two minds with respect to Boids. The first believes that they are an amazing form of procedural AI generation that have been massively underutilized. The second believes they are a useless solution in search of a problem. Personally, I’m somewhere smack dab in the middle. I think the idea is very cool, but in practice I’m not sure I see a use other than as exotic fluff for a bosses projectile attacks, or some novelty enemy behaviour. That’d be nice enough, but more importantly it got me wondering if simpler, more focused force based pathfinding solutions could solve one long standing problem that has plagued EEI from the beginning.

Before I get into that, you may have noticed that I created another YouTube channel. Actually, I accidentally created two, and because Google is a tiny indie company, they won’t let me log into the first one, which also means that I can’t delete it. Yes, I have tried everything, and due to the capricious nature of YouTube censorship, I’ll stick with Odysee videos in the articles. The channel only exists because, eventually, you have to go where the audience is, and that means YouTube and Tiktok. I’m not looking forward to either, but we’ll cross that bridge when we get there.

Anyway, the default way to chase something is to move directly to it. Unfortunately, this behaviour, if programmed into enemies, can be easily beaten through circle strafing. That would be fine, except that swathes of enemies can be trivialized all at the same time, and that’s true even if the enemies are almost as fast as the player. If they’re slow, and especially if they have no ranged attack, they have little hope of even moderately inconveniencing the player.

As a result of this, the main recurring problem with EEI, and just about every other twin stick shooter, is that circle strafing tends to be so overpowered that it can be done mindlessly. In addition to being boring, this really screws with the difficulty curve, as the enemies, trash mobs in particular, effectively do nothing until such time as they build up in big enough numbers as to mathematically overwhelm the player.

I have tried many soft solutions to this problem, which all failed. Hard solutions, such as placing lines of spikes which harshly damage the player, work, yet are relatively undynamic and forced. It’s better than the alternative, but it’s still not great. The ideal is naturalistic enemy behaviour that dynamically entices the player to intelligently move, and utilize all the space in the arena. This should be achievable through an enemy that cuts off the player, but that’s easier said than done.

The most obvious solution is to repel the enemies from the center of the arena. After all, we’re trying to entice the player away from the edges and into the center, so it stands to reason we’d accomplish that by making it less full of damaging enemies. Unfortunately, as shown in the video, it’s not that simple. Enemies often should move from the edges through the center in order to cut off the player. If they are prevented from doing this, they tend to end all being clustered together in one long, squished congo line.

The other simple, obvious solution is to forcefully repel them from each other, with the effect falling off linearly. That helps a bit, but only because the swarm of enemies chasing the player has been artificially spread out, and therefore covers a touch more ground. The same is true for repelling them from the averaged center of all the enemies.

However, that put me onto the real solution. First, we do indeed calculate the center mass of the enemies as a whole. Then, for each enemy, we first calculate the direction towards the player. We pick the direction perpendicular to that which is away from the direction of center mass, and apply a force to the enemy nudging them in that direction. This gives us a constant flanking/surrounding behaviour, while also balancing both sides. It even leads to a really nice behaviour I hadn’t anticipated, which is that enemies curl back, as if they are passing the player off to teammates while playing contain. The only thing to add was a dampening effect when the enemies were squished up against a wall, mainly just to smooth out movement.

Blue == enemies.

I may not have done a great job explaining this algorithm, but once figured out it’s moderately simple. Frustratingly so, considering how much time and energy I spent trying similar things in the past, such as with the riot police. They were supposed to flank the player from both sides. In the end, they sort of got there, but I had to deal with what happens when one of them is stunned, or they’re meleeing the player, or have to use A* pathfinding around obstacles, or any number of issues that aren’t core to the problem. As a result of being too impatient to create a test platform, I ended up dealing with a myriad of additional problems which massively obfuscated the issue. As they say, the lazy man does twice the work.

Now that I’ve figured out the, in hindsight, relatively simple algorithm required to get the anti-circle strafing behaviour I’ve always wanted, the next step is to implement it in-game. I’m picking the P(redditor) trash mob archetype, for obvious reasons, but I’ll save that for the next article.