Can we model human behavior in games with probability?

It’s a really interesting concept to be able to model human behavior in the context of creating a behavioral model for external input beyond the control of a programmer. For a systems designer (games are just human-made systems, after all), it can be really, really interesting! Sometimes, people might put in a weird button combination that can break an important I/O abstraction. In other cases, the programmer might be at fault for failing to account for an erroneous input that crashes the program with the classic segmentation fault.

But for, say, fighting games? Isn’t that a bit strange? Well, no, not really, as this can become an interesting exercise for understanding how to control minds (got your tinfoil hat on?) and optimize the learning experience of any system, even something like an SQL database or a project management system. It’s all the same.

This is from a recently defunct fighting game called Rising Thunder, whose company was recently bought out by Riot Games, the creators of the immensely popular online game, League of Legends.

So, what can we first say about creating such a model? Well, suppose we were to make a model. What would we need?

I’d begin with the relevant ideas and concepts that the subject might possess before he/she/it begins to use our system.

So what would that be for a fighting game? Well, in many 2D fighting games, there are standard conventions or player-game interactions. One example is that most 2D fighters simply have you hold the opposite direction from your opponent to block all incoming strikes. Another one is that simply pressing down on your control pad makes your character crouch. Conventions exist everywhere; every pencil usually has an eraser at its end and most coffee mugs have handles. We take a lot of these for granted but it’s really not; somebody put the time and effort into coming up with these and a lot of other people also put the effort in to follow them.

In fighting games, there is also the concept of balance and power. In short, power represents the “strength” of a specific character and their moveset that allows them to perform better during games, either by being able to dish out more damage in a combo or having some sort of psychological concept that erodes the other player’s mind. Balance represents the relative strength of characters to one another. For example, if one character can kill all the other ones with one hit and no other character can do that, that character and their killing move is not balanced unless it has a downside to it.

In many fighting games, a “balance” triangle of moves like this usually occurs. If a player is blocking, you grapple them. Grappling? Make them miss and attack them instead. And, finally, block strikes.

Now, suppose we only had these three moves in our fighting game. No moving too! This means that the entire game is based on just 3 moves: Striking, Blocking, and Grappling.

Suppose we want to design characters in this system. How can we design characters that are balanced?

Well, suppose we were to design a character that always did 10 hitpoints when they successfully struck the other person, and that every character had 10 or less hitpoints. That’s a one-hit kill! But if that other player were to always block, then that character, despite all of their power, would be practically useless.

Of course, we think that nobody’s going to block forever. But why? Because nobody wins in a fighting game by not actually fighting. In most games, only striking and grappling does damage.

Already, we see that our fighting game is more than just rock-paper-scissors–there is already an imbalance that players would prefer striking and grappling over blocking simply by the sheer notion that blocking does not directly contribute to winning the game by doing damage.

The real question: do players block less often because blocking does not win any games?

This is the real question: its answer determines if a definite value for “balance” can ever be decided.

Already, we can begin to establish a mathematical basis for analyzing the game.

P(blocking) = ?

Ideally, the probability of all moves would look like this:

P(striking) + P(blocking) + P(grappling) = 0.33 + 0.33 + 0.33 = about\ 1

In reality, we know that as the probabilities of one player’s moves become skewed, so does the other once they adapt to the move style of the other player.

Such probabilities are not static–they are in flux and change as the situation changes. But we know that they are dependent on something; at least, the patterns that one might perceive of the other player.

But, indeed, the uneven “landscape” of the different move probability distributions forms what we call “the meta” in certain types of games.

The patterns of move or choice distributions given a specific situation in a game, their “values,” and the optimal strategies to take advantage of such distributions eventually form what we call “the meta” in many types of games.

Even for games like Chess, there is a “meta.” The opening moves for chess can be varied, but people probably won’t want to take the move that you get you checkmate’d in less than 5 moves. Even knowledge of the basic move tree can affect the way people play a game, already muddying the waters.

Now, we understand that the probability of some a player taking some actions depends on what they think they should do. In fact, the internal valuation model that a player has contributes to how they calculate the value of their next move; in a way, each player should have their own model which calculates the “utility” of each of their actions in a game. Even the “absence” of a such a model is important, as their actual behavior might be decided by some sort of “default” model that most “under-developed” players exhibit; for example, fighting game players tend not to understand the strike-block-grab triangle and fail to produce a grappling counter to a more skilled opponent who blocks without repercussion. But what causes players to gravitate certain styles of play or patterns?

In fact, could we simply devalue the notion of “probability” entirely, and instead, opt for a notion of utility? In a game with sentient players who understand the rules and how to optimize their own actions towards certain rules, one cannot simply adopt a uniform, static probability distribution for modeling human behavior. What is a player action probability distribution if the parameters that decide it are always changing, and the properties of the data are always changing? Would it instead be better to use a non-smooth function as a model for calculating a probability substitute?

So now the question is: Is there a way to translate a meta-game into a probabilistic model, and is it useful? What would be a suitable substitute?

It’s really strange to try and create a useful model; something as simple as a straight-up probabilistic model with uniform distribution might prove inefficient. Suppose one made a function model for a game of rock-paper-scissors; it’s quite easy to determine the best move given a certain probability distribution–play paper if your opponent is going to play rock. Suppose a player plays rock three times in a row. A person might think, “Oh, the chance of my opponent playing rock again is low, so I should opt for paper to beat it!” But, in reality, the other player could predict that the other player is going to play paper and instead play scissors to counter the counter. And then what if the original player predicts that? And what is the probability of each case happening? Then, I suppose, we start edging into the inner makeup of the minds of each player, trying to categorize them into maybe even different personality types with certain proclivities to read into situations 2 layers rather than 3. But what if a player knew that? Then perhaps they would read 1 or 4 times deep into the situation to counter that piece of knowledge.

Really, even if we try to constrain ourselves to simple probability, the number of variables that can contribute to those probabilities start to explode and suddenly we just have too much to deal with. Can we even measure the probability that a person reads 10 times into a situation, and then the probability that a person thinks they should read 10 times deep?

One must wonder; really, if knowledge alone can heavily define our actions; the absence of a crucial piece of information can really determine if a player will really do something. A player that is not even aware they can do something, like grapple in our fighting game example, really won’t be represented accurately if we take something and use a uniform distribution, as they’ll never consider even performing the action, and would only do it by accident.

Even better, can knowledge even be treated as a discrete variable? Though its simple to say something like, “Oh, I know I can grapple now!” and then model a player’s behavior as closer to a uniform distribution, it’s really strange to simply only have a system when things can be “on” and “off” if we consider games that have imperfect knowledge–“fog-of-war”-type games–and then try to categorize a player trying to calculate the probability of a certain action happening and then taking an action as “zero” or “one.”

Perhaps something from decision theory would instead be a better substitute. It would be good to dive into topics to explore the range of possibilities to illustrate this problem.