Help me test out my Bayes Academy game
You may remember my earlier posts on a game that would teach you Bayesian networks. As might be expected, the original plans turned out to be way too ambitious and I ended up cutting a lot of the story content and turned into more of a simple puzzle game, but here it is regardless.
I need people to test this out to get material for my thesis. So if you could fill out one brief questionnaire, play the game for a bit, and then fill out another questionnaire, that’d help a lot. You may also learn a bit in the process—at least for me, I’ve found that it made concepts like the conjunction fallacy and value of information clearer. Here’s a link to the first questionnaire, and you’ll get the link to the actual game once you fill out the form. Thanks a lot! [Java required.]
Also please let me know if you’re having any trouble with it.
ETA: Known issues: Some Mac users have reported an issue where the game info panel vanishes after a few steps. I’ve updated the JAR with a version that has the info buttons at the top; this won’t help you complete the tutorial, but at least it might be possible to play the actual game (though I’m not sure of how easy it will be to understand without the tutorial). When you’ve hit “done”, the graph will change colors to show which variables you guessed correctly; at that point you need to hit space which *hopefully* should work even though you can’t see the text at the bottom.
ETA2: The game has now been updated with a version that lets you choose a lower resolution setting, which should help for people who were having problems with some nodes going off the screen / the text being too large.
I actually really like that you have to spend a resource to learn new information and that the score is dependent on luck. I.e. you use limited resources to optimize the gamble you are making. That seems like a very good description of how life works, only, it’s all transparent and quantified in your game.
Some suggestions:
In the tutorial, why do I first get to read a description of a picture and then I’m presented with the picture? Obviously, it should be the other way around.
You should be able to progress the text by mouse.
It should be easier to distinguish new text from old. I think in visual novels, the text box never “scrolls”. If the new text doesn’t fit into the text box, or to make a new paragraph, the text box is cleared. You could make separate textboxes for the current message and the history.
The confusing notation is a real distraction and zaps away a lot of the potential fun. Understanding the notation actually seems more interesting than winning the game, but I have too little information to understand it, which leads to frustration. Why are there two big boxes with normal nodes? Why do normal nodes have all those boxes instead of simple bar that shows the probability? Why do bayes-nodes have all those rows instead of just two bars? What are there grey bars? How do ‘and’ and ‘or’ nodes work? I would think that one input corresponds to the vertical division and one input to the horizontal division. It should be more obvious which node is which (by having the input into that side of the box). The connections of nodes did not have arrows. If I understood the game correctly, that would help distinguish inputs from outputs.
The effect of clicking on one node shouldn’t be instant. At first, it should probably go step by step: You click on a node and reveal it’s truth-value (some text appears explaining which node changed and why). Press a key → the next affected node gets updated. Until affected nodes are updated. Later you don’t have to click, there is a small pause between each change. That way you could see the effect of measuring a node and understand why the effect was the way it was, instead of … trying to work that information out for yourself with only being able to see the aftermath.
You should make it more linear. Put the tutorial and the main game into one. I don’t see the use of this decision between introductory and intermediate psychology.
Have the player start out with much simpler networks and infinite energy.
Introduce new types of nodes during the game, not all at once in the tutorial. Every time you introduce something new, go back to simple networks with unlimited energy.
Of those, explaining or simplifying the notation seems the most important to me.
Thanks! I agree with all of your points and had considered implementing many of them myself: unfortunately, while working on this project I learned that I hate UI programming, and finally got to the point where I just wanted to put out a not-too-totally-horrible prototype and be done with it. :(
The source code was written to employ a bit of an MVC architecture, with the intention of making it easier for other people to implement a better UI afterwards… but in retrospect just rewriting the whole thing under a better platform than Java might be the best approach, if anyone wants to do that.
I tried it on Ubuntu. The game is practically unplayable. I only see the last line of the text unless I scroll, and most of the bottom box is covered. Is the text supposed to be so huge?
I’ve just uploaded a new version that lets you choose a lower resolution setting (and thus a smaller font size). Sorry about that.
I’m finding the game … not fun. Here are some reasons.
The notation for “Bayes nodes” is confusing and figuring out what it’s saying requires more conscious thought than I really want to be putting into a game like this.
There’s this menu where I have to choose “go to lectures”, “study in the library”, or “hang out with friends” all the time—except that only one choice is actually ever available (so far, at least).
There’s too much now-press-space, somehow.
In the actual node-guessing game that’s the main activity, you usually get to inspect about one node per lecture on average. That’s not enough to make it very interesting. (Also: if the idea is to teach people about separation, so that they can figure out when knowing the state of one node means that learning X doesn’t tell you anything about Y, wouldn’t it be better if that situation were to arise more often?)
It doesn’t feel to me as if any sort of highbrow analysis of the networks provides a substantial advantage, which is a shame if the idea is to make people learn to do it. On the other hand …
So far I’ve played three or four games and been rejected by the academy about halfway through every time. So maybe there are strategic lessons I haven’t learned yet. Or maybe it’s just that success is too dependent on luck?
Thanks for the feedback, I agree with a lot of your criticisms.
Re: the difficulty: just played the game and thrice and failed on each attempt. Huh, I remember that when I was trying to get the difficulty right before releasing it, I managed to get pretty reliably to the last stages, if not beat it. Looks like I notched the difficulty too up, when it used to be too easy at first.
I ran it on Windows, at work. It took over my entire monitor, with no sign of any way to make it run inside an ordinary window. Nope nope nope. I might try it later at home.
Not about the game itself, but the wording of the questions is a bit confusing to me:
The act of observing a variable doesn’t influence any of the variables, it would only change your beliefs about the variables. The only things influencing a variable are its parents in the Bayesian network.
I had a similar issue. Just what was meant by “influence”? I think it had to be “change my beliefs about”.
Who’s your target audience/guinea pigs? Most normal people would run away screaming after seeing the “This survey tests your understanding of d-separation in Bayesian networks” part.
Well, I was hoping the “don’t worry if you don’t understand the questions” bit to help avoid the running away effect. :)
Ok, I wanted to play it, but the word are too big and/or the textbox is too small. graphics don’t run well, and it really needs an exit button. I’m using windows 8. other than that, I think the idea is great.
I’ve just uploaded a new version that lets you choose a lower resolution setting (and thus a smaller font size). Sorry about that.
Awesome! What’s the download link without retaking the survey?
http://kajsotala.fi/Thesis/BayesGame.jar / http://kajsotala.fi/Thesis/BayesGameWindows.rar
(posting direct links here rather than send a PM to minimize trivial inconveniences for anyone else who might have taken the survey already, but if you’re reading this and haven’t, please do take the survey before playing—and then answer the posttest survey after playing, too :) )
I’m still getting a weird big screen with a tiny little text area at the bottom. Is that how it’s supposed to be?
Nope. :(