Teaching Bayesian networks by means of social scheming, or, why edugames don’t have to suck

[xuenay]

As a part of my Master’s thesis in Computer Science, I am designing a game which seeks to teach its players a subfield of math known as Bayesian networks, hopefully in a fun and enjoyable way. This post explains some of the basic design and educational philosophy behind the game, and will hopefully also convince you that educational games don’t have to suck.

I will start by discussing a simple-but-rather-abstract math problem and look at some ways by which people have tried to make math problems more interesting. Then I will consider some of the reasons why the most-commonly used ways of making them interesting are failures, look at the things that make the problems in entertainment games interesting and the problems in most edutainment games uninteresting, and finally talk about how to actually make a good educational game. I’ll also talk a bit about how I’ll try to make the math concerning Bayesian networks relevant and interesting in my game, while a later post will elaborate more on the design of the game.

So as an example of the kinds of things that I’d like my game to teach, here’s an early graph from the Coursera course on Probabilistic Graphical Models. For somewhat mathy people, it doesn’t represent anything complicated: there’s a deterministic OR gate Y that takes as input two binary random variables, X1 and X2. For non-mathy people, that sentence was probably just some incomprehensible gibberish. (If you’re one of those people, don’t worry, just keep reading.)

I’m not going to go through the whole example here, but the idea is to explain why observing the state of X1 might sometimes give you information about X2. (If the following makes your eyes glaze over, again don’t worry – you can just skip ahead to the next paragraph.) Briefly, if you know that Y is true, then either X1 or X2 or both must be true, and in two out of three of those possible cases, X2 is true. But if you find out that X1 is true, then that eliminates the case where X1 was false and X2 was true, so the probability of X2 being true goes down to .5 from .66. In the course, the explanation of this simple case is then used to build up an understanding of more complicated probabilistic networks and how observing one variable may give you information about other variables.

For mathy types the full explanation is probably relatively easy to follow, at least if you put in a little bit of thought. But for someone who is unfamiliar with math – or worse, scared of it -, it might not be. So the question is, how to convert that explanation to a form that is somewhat easier to understand?

The traditional school math approach would be to convert the abstract explanation into a concrete “real-life” case. Let’s say that the variables are people. X1 becomes Alice, X2 becomes Bob, and Y becomes Charlie. A variable being true means that the person is question has heard about some piece of information – say, that Lord Deathfist the Terrible is on a rampage again. If one takes the lines to mean “Alice tells Charlie stuff and Bob tells Charlie stuff (but Alice and Bob don’t talk with each other)”, the “OR gate” thing becomes relatively easy to understand. It means simply that Charlie knows about the rampage if either Alice or Bob, or both, know about it and have told Charlie.

Now we could try to explain it in common-sense terms like this: “Suppose that Charlie knows about Lord Deathfist. That means that either Alice, or Bob, or both, know about it, and have told him. Now out of those three possibilities, Alice knows about it in two possible cases (the one where Alice knows, and the one where Alice and Bob both know) and there’s one possible case where she does not know (the scenario where only Bob knows), so the chance of Alice knowing this is ⅔. But if we are also told that Bob knows it, that only allows for the possibilities where 1) Bob knows and 2) both Alice and Bob know, so that’s 1 possibility out of 2 for Alice knowing it, so the chance of Alice knowing goes down to ½ where it used to be ⅔.”

This is… slightly better. Maybe. We still have several problems. For one, it’s still easy to lose track of what exactly the possible scenarios are, though we might be able to solve that particular problem by adding animated illustrations and stuff.

But still, the explanation takes some effort to follow, and you still need to be motivated to do so. And if we merely dress up this abstract math problem with some imaginary context, that still doesn’t make it particularly interesting. Who the heck are these people, and why should anyone care about what they know? If we are not already familiar with them, “Alice” and “Bob” aren’t much better than X1 or X2 – they are still equally meaningless.

We could try to fix that by picking names we were already familiar with – like Y was Luke Skywalker, and X1 and X2 were Han Solo and Princess Leia, and Luke would know about the Empire’s new secret plan if either Han or Leia had also found out about it, and we wanted to know the chance of all of them already knowing this important piece of information.

But we’d still be quite aware of the fact that the whole Star Wars gimmick was just coating for something we weren’t really interested in. Not to mention that the whole problem is more than a little artificial – if Leia tells Luke, why wouldn’t Luke just tell Han? And even if we understood the explanation, we couldn’t do anything interesting with it. Like, knowing the logic wouldn’t allow us to blow up the Death Star, or anything.

So some games try to provide that kind of significance for the task: work through an arithmetic problem, and you get to see the Death Star blown up as a reward. But while this might make it somewhat more motivating to play, we’d rather play an action game where we could spend all of our time shooting at the Death Star and not waste any time doing arithmetic problems. Additionally, the action game would also allow us to shoot at other things, like TIE Fighters, and that would be more fun.

Another way of putting this would be that we don’t actually find the math task itself meaningful. It’s artificial and disconnected from the things that we are actually interested in.

Let’s take a moment to contrast this to the way that one uses math in commercial entertainment games. If I’m playing XCOM: Enemy Unknown, for instance, I might see that my enemy has five hit points, while my grenade does three points of damage. Calculating the difference, I see that throwing the grenade would leave my enemy with two hit points left, enough to shoot back on his turn. Fortunately I have another guy nearby, and he hasn’t used his grenade either – but I also know that there are at least six more enemies left on the battlefield. Do I really want to use both of my remaining grenades, just to take out one enemy? Maybe I should just try shooting him… both of my guys have a 50% chance to hit him with their guns, and they’d do an average of three points of damage on a hit, so that’s an expected three points of damage if both take the shot, or – calculating it differently – a 25% chance of killing the alien dead… which aren’t very good odds, so maybe the other guy should throw the grenade and the other shoot, and since grenades magically never miss in this game, I’d then have a 50% chance of killing the alien.

So as I play XCOM, I keep running arithmetic calculations through my head. But unlike in the “solve five arithmetic problems, then you get to see the Death Star blowing up” example, these calculations aren’t just glued-on. In fact, while playing, I never actually think that I am solving a set of arithmetic and probability problems in order to be rewarded with the sight of the enemies dying and my soldiers surviving. I think that I’m out killing aliens and doing my best to keep my guys alive. (How many of you had realized that XCOM is an educational game that, among other things, drills you on arithmetic problems? Well, it is!)

This can be a bad thing in some senses – it means that I’m engaging in “stealth learning”, learning a skill without realizing it. Not realizing it means that I can’t consciously reflect and introspect on my learning, and I may have difficulties transferring the skill to other domains, since my unawareness of what I’m doing makes it harder to notice if I happen to run across a problem that employs the same principles but looks superficially different. But it does also mean that the calculations are very much meaningful, and that I don’t view them as an unnecessary annoyance that I’d rather skip and move on to the good parts.

The game scholars Katie Salen and Eric Zimmerman write:

Another component of meaningful play requires that the relationship between action and outcome is integrated into the larger context of the game. This means that an action a player takes not only has immediate significance in the game, but also affects the play experience at a later point in the game. Chess is a deep and meaningful game because the delicate opening moves directly result in the complex trajectories of the middle game-and the middle game grows into the spare and powerful encounters of the end game. Any action taken at one moment will affect possible actions at later moments.

The calculations in XCOM are meaningful because they let me predict the immediate consequences of my choices. Those immediate consequences will influence the outcome of the rest of the current battle, and the end result of the battle will influence my options when I return to the strategic layer of the game, where my choices will influence how well I will do in future battles…

In contrast, the arithmetic exercises in a simple edutainment game aren’t very meaningful: maybe they let you see the Death Star blowing up, but you don’t care about the end result of the calculations themselves, because they don’t inform any choices that you need to make. Of course, there can still be other ways by which the arithmetic “game” becomes meaningful – maybe you get scored based on how quickly you solve the problems, and then you end up wanting to maximize your score, either in competition with yourself or others. Meaning can also emerge from the way that the game fits into a broader social context, as the competition example shows. But of course, that still doesn’t make most edutainment games very fun.

So if we wish people to actually be motivated to solve problems relating to Bayesian networks, we need to embed them in a context that makes them meaningful. In principle, we could just make them into multistage puzzles: DragonBox is fantastic in the way that it turns algebraic equations into puzzles, where you need to make the right choices in the early steps of the problem in order to solve it in the most efficient manner. But while that is good for teaching abstract mathematics, it doesn’t teach much about how to apply the math. And at least I personally find games with a story to be more compelling than pure puzzle games – and also more fun to design.

So I’ll want to design a game in which our original question of “does Bob also know about this” becomes meaningful, because that knowledge will inform our choices, and because there will be long-term consequences that are either beneficial or detrimental, depending on whether or not we correctly predicted the probability of Bob knowing something.

My preliminary design for such a game is set in an academy that’s inspired both by Harry Potter’s Hogwarts (to be more specific, the Hogwarts in the fanfic Harry Potter and the Methods of Rationality) and Revolutionary Girl Utena’s Ohtori Academy. Studying both physical combat and magic, the students of the academy are a scheming lot, ruled over by an iron-fisted student council made up of seven members… And figuring out things like exactly which student is cheating on their partner and who else knows about it, may turn out to be crucial for a first-year student seeking to place herself and her chosen allies in control of the council. If only she can find out which students are trustworthy enough to become her allies… misreading the evidence about someone’s nature may come to cost her dearly later.

In my next post, I will elaborate more on the preliminary design of the game, and of the ways in which it will teach its players the mathematics of Bayesian networks.

Originally published at Kaj Sotala. You can comment here or there.