# Rationality Games & Apps Brainstorming

Last month, mo­bile gam­ing su­per­star An­gry Birds was out-sold in some coun­tries by DragonBox, a kids game in which play­ers solve aleg­bra equa­tions.

How does the game work? Jonathan Liu ex­plains:

There are five “wor­lds,” each with twenty lev­els, and as you progress through the lev­els the “drag­ons” hatch and grow into their full-sized ver­sions. While this in it­self has noth­ing to do with alge­bra, I men­tion this be­cause my kids love this. It’s a very tiny in­cen­tive (along with earn­ing stars) but they re­ally want to beat the next level to watch the dragon grow into its next form. I was told that the drag­ons were all drawn by a four­teen-year-old girl, and they’re a lot of fun. (They aren’t all typ­i­cal drag­ons — One starts off more like a fish, one looks like a squid, and so on.)

You are pre­sented with a big screen with two trays, each con­tain­ing a num­ber of “cards” with differ­ent images on it. Some­where on the screen there will be a lit­tle box with a star on it, sparkling and glow­ing. The app gives very min­i­mal in­struc­tions in a hand-writ­ten font with ar­rows point­ing to rele­vant spots on the screen, but it tells you to get the box by it­self. At first you do this sim­ply by tap­ping the green spirally cards, which van­ish when you tap them. Then, you’ll start to get some “night” ver­sions of cards — drag these onto the “day” ver­sions and they be­come green swirls, which you already know how to han­dle.

After you’ve got­ten past sev­eral lev­els of mov­ing cards around and tap­ping on swirls, you’ll get a few cards down at the bot­tom which you can drag onto the trays — but when­ever you drag a card onto one side, you have to also drag a copy to the other side as well. (This, of course, simu­lates adding the same num­ber to both sides.) And then, a few lev­els on, you learn that you can flip these ex­tra cards from day to night (and vice versa) be­fore drag­ging them onto the trays.

As the game pro­gresses, you’ll start see­ing cards that are above and be­low each other, with a bar in the mid­dle — and you’ll learn to can­cel these out by drag­ging one onto the other, which then turns into a one-dot. And you’ll learn that a one-dot van­ishes when you drag it onto a card it’s at­tached to (with a lit­tle grey dot be­tween them). Th­ese, of course, are frac­tions — mul­ti­pli­ca­tion and di­vi­sion — but you don’t need to know that to play the game, ei­ther.

The key to DragonBox’s suc­cess is not that it’s the best alge­bra tu­to­rial available, but rather that it’s ac­tu­ally fun for its tar­get au­di­ence to play.

Others have no­ticed the po­ten­tial of “com­puter-as­sisted ed­u­ca­tion” be­fore. Aubrey Daniels writes:

When you an­a­lyze [video games], you will see that the player is clear about what is ex­pected of him, that [the] player’s be­hav­ior is con­tinu­ally mea­sured, and [that] the player is pro­vided with feed­back so he knows what the mea­sure­ments re­veal about his perfor­mance. Fi­nally, and most im­por­tantly, as he plays the game, the player re­ceives high rates of re­in­force­ment which mo­ti­vate him to play the game over and over again. In fact, re­in­force­ment oc­curs up to 100 times a minute.

Re­mem­ber what works in re­in­force­ment: Small re­in­force­ments are fine, but the re­in­forcer should im­me­di­ately fol­low the tar­get be­hav­ior, and it should be con­di­tional on the spe­cific be­hav­ior you want to strengthen.

Video games are perfect for that! Lit­tle hits of re­in­force­ment can be given many times a minute, con­di­tional on ex­actly the kind of be­hav­ior your want to re­in­force, and con­di­tional on ex­actly the be­hav­ior you want to re­in­force.

DragonBox is just a par­tic­u­larly suc­cess­ful im­ple­men­ta­tion of this in­sight.

One of the goals for the Cen­ter for Ap­plied Ra­tion­al­ity is to de­velop ra­tio­nal­ity games and apps. But it’s tricky to think of how to make ad­dic­tive games that ac­tu­ally teach ra­tio­nal­ity skills. So I’d like to provide a place for peo­ple to brain­storm ideas about what would make an ad­dic­tive and in­struc­tive ra­tio­nal­ity game.

See also: Ra­tion­al­ity and Video Games, Gam­ifi­ca­tion and Ra­tion­al­ity Train­ing, Raytheon to Develop Ra­tion­al­ity-Train­ing Games.

• This was (more or less) the dis­cus­sion topic at the last Toronto meetup. Here’s what we dis­cussed (NOTE: these are min­utes of a LW meetup so don’t ex­pect it to be 100% on-topic). Also see the wiki page with pre­vi­ous dis­cus­sion threads

No defini­tive game idea yet, but lots of in­ter­est­ing sug­ges­tions came up.

Harry Pot­ter & the Meth­ods of Ra­tion­al­ity: The Game.

• Sugges­tion was not to play Harry, as he already knows ev­ery­thing (or a lot, any­way). In­stead you have to deal with Harry.

Real­time stat­egy game where you re­cruit units rather than build­ing them

• ei­ther bully or befriend units or just make your team the most fun (Draco/​Hermione/​Harry strate­gies)

• can also just pay units but that ob­vi­ously means you need to ac­quire more resources

• units are al­igned to differ­ent fac­tions, and you can sig­nal loy­alty to one fac­tion to gain their sup­port at the ex­pense of the other

Epistemic ra­tio­nal­ity: the game

• dis­cover how the game me­chan­ics work as you go along

• Is it pos­si­ble to pro­ce­du­rally gen­er­ate the laws of physics? (e.g. num­ber of di­men­sions, grav­i­ta­tional con­stant etc. ran­domly gen­er­ated)

• I was wor­ried that most com­bi­na­tions of phys­i­cal laws would be un­playable for one rea­son or the other, and un­like real life we don’t have the an­thropic prin­ci­ple to help us out

• game of sci­ence, e.g. build­ing atoms into molecules with differ­ent laws of physics

Ex­treme cooperation

• Both play­ers con­trol­ling same character

• (Alien hand syn­drome)

• The Elephant and its Rider: The Game. One player as­signed the role of ra­tio­nal­is­ing the other’s be­havi­our.

• The op­po­site of this is a sin­gle player con­trol­ling two char­ac­ters but with the same con­trols (e.g. you can’t move one char­ac­ter right with­out mak­ing the other char­ac­ter fall off a cliff). Not re­ally LWish but might be fun.

Bayes The­o­rem game

• Mur­der mys­tery/​court case

• Base on real life court cases?

• Game where you have to pro­gram your own rewards

• The way game­play is set up, you are mo­ti­vated to achieve far goals but not near ones

• Game­play is too frus­trat­ing un­less you can cal­ibrate vi­sual re­wards so that you get re­warded for do­ing vaguely the right thing

• This was my idea but I don’t know whether the con­cept even makes sense

Game teaches you real world stuff in­ci­den­tally.

NPC’s prone to differ­ent biases

• First use your Bayesi­an­ness to work out who has which bias

• Then work out how to use char­ac­ters’ bi­ases to defeat them or per­suade them to join your side

• Can we simu­late bi­ased player also?

• Close off el­e­ments of the di­alogue tree de­pend­ing on what your bi­ases are sup­posed to be.

• (My idea for a re­pro­gram­ming your own brain game. More pre­cisely, re­pro­gram­ming in­ter­face be­tween in­put de­vices and what your char­ac­ter does. Not re­ally LW)

Time Portal

• Like Por­tal ex­cept jump­ing through the por­tal dis­places you in time (one way is for­wards, the other back­wards)

• Again not re­ally LWish.

Fix moral system

• Ex­ist­ing games tend to be kick puppy vs. feed puppy

• Some­times you should get good re­sults from bad actions

• Trol­ley prob­lem: the game

Game that starts off like the Sims and ends up like Civilization

• Weapon­is­ing ap­par­ently in­no­cent game me­chan­ics, e.g. steal­ing all the fire alarms from some­one’s house and then cook­ing something

Ex­ist­ing games which we like and/​or which came up in the dis­cus­sion:

• Braid

• Portal

• Limbo

• Fez

• Phoenix Wright

• Psychonauts

• A flash game called Chronotron

• Sid Meier’s Alpha Centauri

• Some game which was like Ikaruga but co­op­er­a­tive?

• Mass Effect

• A while ago I thought it’d be pretty neat to have a MMORPG based around in-game maths/​physics/​pro­gram­ming/​logic puz­zles. Kind of like a dy­namic Pro­ject Euler, but with a pret­tier steam­punk-hip­ster-nar­ra­tive front-end, and in­stead of lev­el­ling up by gain­ing some ab­stracted mea­sure of ex­pe­rience, you lev­el­led up by ac­tu­ally get­ting good at do­ing stuff.

I’m mostly throw­ing this idea out there so some­one else doesn’t have to, not be­cause I think it’s ac­tu­ally a good idea. I strug­gle to think up a way a ra­tio­nal­ity-based MMORPG would be fun to play.

• Product rule skill level 2 achieved! New skill un­locked: Chain rule!

A wild com­pos­ite func­tion ap­pears!

• *dies* Didn’t you hear ev­ery­one yel­ling at you to stop us­ing the chain rule? If you’d read the wiki page be­fore the fight, you’d have known that it draws ag­gro like mad in this dun­geon. That’s why ev­ery­one but the tank was prac­tic­ing with limits last night.

• 9 Jul 2012 16:15 UTC
5 points

Psuedo-In­trade. Use a fake cur­rency, but oth­er­wise keep it ba­si­cally iden­ti­cal. Play­ers can use their not-money to buy worth­less-but-pretty tro­phies and what­not for their Throne Rooms of Ra­tion­al­ity. Prob­a­bly in­clude some oc­ca­sional free money to keep the game above zero-sum, which could be dis­cour­ag­ing (i.e. cre­ate bots and give them free money to place on bad bets).

• Cal­ibrated Pro­ject Euler

(for pro­gram­mers only)

Have a set of lit­tle pro­gram­ming/​maths ex­er­cises, like Pro­ject Euler or some pro­gram­ming challenges. -things like “re­turn the num­ber of prime num­bers in a list”, “find the longest in­creas­ing sub­se­quence in a list”, etc.

First you give an es­ti­mate for how long it will take to write a solu­tion, then you write your solu­tion (in the app it­self), then you give an es­ti­mate of how likely it is that your solu­tion is cor­rect, and then your solu­tion is ex­e­cuted and you see whether it works or not.

This could help miti­gate the plan­ning fal­lacy and over­con­fi­dence and al­lows pretty quick iter­a­tions, but only works for pro­gram­mers.

• This is a good idea, but might be gen­er­al­ized and sim­plified. Sim­ply put down a task and the time you think it will take you to com­plete it. This is then pub­lished pub­li­cly or semi-pub­li­cly—the key thing is that some­one (prob­a­bly an­other user) can now ver­ify when you’ve com­pleted a task.

Sim­pler, ap­pli­ca­ble to any kind of task, and cal­ibrated by a third-party.

• True, but that re­quires ded­i­ca­tion; the triv­ial in­con­ve­niences of the sys­tem for pub­lish­ing the task etc. means you’ll of­ten just skip it. Though it is the kind of thing that could be in­te­grated in a todo list or some­thing; for ex­am­ple a todo list soft­ware that de­tects when an item has been on the list for say two days, and asks you for a con­fi­dence in­ter­val for when you ex­pect to do it.

An ad­di­tional benefit of a ded­i­cated app/​game is that you can in­te­grate things like leader­boards, you can com­pare your­self to oth­ers, etc. - com­pe­ti­tion can be quite the mo­ti­va­tor.

Have you ever done Google Code Jam? Some­thing like that back­end would be good for solu­tion sub­mis­sion and check­ing, while still al­low­ing you to pro­gram with what­ever the hell you want to.

• I won­der if it would be pos­si­ble to build a wa­ger­ing/​prob­a­bil­is­tic Zendo cross­breed. That is, the com­puter is will­ing to be Dutch-booked, if you can only cor­rectly es­ti­mate the prob­a­bil­ities given some ex­am­ples. You might even be able to make sce­nar­ios rep­re­sent­ing var­i­ous failures of ra­tio­nal­ity, like the Linda ex­am­ple (“green is more likely than red; stars than tri­an­gles; smil­ing than frown­ing; bounc­ing than glow­ing—now, which is more likely: the star, or the green, bounc­ing, smil­ing star?”, or the 2-4-6 case, or maybe even the Availa­bil­ity heuris­tic (the sys­tem will be in­clined to show you ex­am­ples where you made a lot of money, in con­texts where bet­ting on them would lose you money).

• Record­ing the set of one’s past games would help a lot with re­liev­ing the availa­bil­ity heuris­tic.

• 9 Jul 2012 16:17 UTC
4 points

A game where play­ers race to de­sign a self-im­prov­ing AI. Win­ner gets free pa­per­clips.

• A differ­ent kind of men­tal train­ing via gam­ing is available via lu­mos­ity.

• I just tried out DragonBox, and it is quite fun. It’s a pretty poor in­struc­tional aid, though. Sure, it teaches you how to bal­ance equa­tions by fol­low­ing the cor­rect rules, but it doesn’t ex­plain where the rules come from. Alge­bra is more than rote mem­o­riza­tion; a stu­dent who truly un­der­stood alge­bra would be able to re-cre­ate all the rules, such as “if you add some­thing to the left side, add it to the right side as well”, com­pletely from scratch. DragonBox does not teach this level of un­der­stand­ing.

• Just be­cause some­thing doesn’t teach ev­ery­thing about a sub­ject doesn’t make it a poor in­struc­tional aid. Alge­bra is more than rote mem­o­riza­tion, but hav­ing the rules mem­o­rized does help a lot in get­ting to the stage where you truly un­der­stand it. Even if you do un­der­stand it and could in prin­ci­ple recre­ate the rules from scratch, re­call­ing them from mem­ory is faster than de­riv­ing them each time you need them—and since alge­bra is pretty much the foun­da­tion of all ad­vanced math, you’ll be need­ing them a lot if you want to study math at all. (Though if you end up strug­gling with the harder top­ics be­cause you didn’t have the rules of alge­bra ap­pro­pri­ately mem­o­rized, you might never want to study more of it...)

• Alge­bra is more than rote mem­o­riza­tion, but hav­ing the rules mem­o­rized does help a lot in get­ting to the stage where you truly un­der­stand it.

I dis­agree. I think that mem­o­riz­ing the rules first, with­out un­der­stand­ing where they come from, dis­cour­ages the stu­dent from at­tempt­ing to un­der­stand any­thing to be­gin with. After all, his goal is to bal­ance an equa­tion, and look, he just bal­anced it… so what else is there to know ? Thus, the mem­o­riza­tion ap­proach cre­ates the im­pres­sion that math (or what­ever sub­ject you’re study­ing) is all about ar­bi­trary rules that make no sense; it’s all about “guess­ing the teacher’s pass­word”, and that’s bor­ing.

Con­trast this with the ap­proach of treat­ing an equa­tion like a puz­zle. If “2x − 3 = 5”, and we want to know what x is, there are many ways to ap­proach the solu­tion. We could ask, “some­one did a bunch of stuff to x to get 5, how can we undo it ?”, or we could say, “the equa­tion is like a pair of scales that are bal­anced, so what can we do to get x by it­self with­out un­bal­anc­ing the scales ?”, etc. Some pos­si­ble par­tial an­swers are, “some­one took away 3, so let’s add it back”, or “if we add 3 to both sides, the scales will still be bal­anced but we’ll be one step closer to a solu­tion”. But “add 3 to both sides be­cause that’s how the game is pro­grammed and you won’t get the high score oth­er­wise” isn’t much of an an­swer. High scores don’t mean any­thing, alge­bra does.

• Well, I can’t speak for oth­ers, but my per­sonal ex­pe­rience with math tends to be that I only start prop­erly learn­ing why some­thing works once I have the rules pretty well mem­o­rized. Be­fore that, my work­ing mem­ory is so oc­cu­pied with try­ing to just re­mem­ber how to ap­ply the rules that I don’t have the space to re­mem­ber why they work. Or al­ter­na­tively, I can learn why the rules work—but in that case I don’t have the mem­ory ca­pac­ity left for re­mem­ber­ing how to ap­ply them.

Of course, this is com­pli­cated by the fact that dur­ing the pro­cess of try­ing to mem­o­rize the rules, I of­ten stop to think about why they work in an at­tempt to red­erive them and make sure I’m not mis­re­mem­ber­ing them. So it’s not pure rote mem­o­riza­tion, like the way it seems to be with DragonBox. But I would still ex­pect that if some­body first learned them as mean­ingless rules in the game, and was then later taught math and the rea­sons for the rules, they’d have a good chance of be­ing delighted at dis­cov­er­ing where the rules came from, and could spend all of their cog­ni­tive ca­pac­ity on de­vel­op­ing an ac­tual un­der­stand­ing.

• Fair enough; it’s pos­si­ble that you and I sim­ply think in differ­ent ways. I per­son­ally find it very difficult to mem­o­rize (seem­ingly) ar­bi­trary rules, and I found it very difficult to un-teach the “guess the teacher’s pass­word” men­tal­ity to peo­ple. But it’s quite likely that I’m mak­ing an un­jus­tified gen­er­al­iza­tion from a very small num­ber of ex­am­ples.

I won­der if there’s any lay­man-ac­cessible liter­a­ture on this topic...

• Time es­ti­ma­tion app

At the be­gin­ning of each week, en­ter the things you plan to do dur­ing the week, the prob­a­bil­ity that you es­ti­mate of do­ing them, and the per­centage of time you’ll spend on var­i­ous dis­trac­tions (eg. if you use Res­cue­time to count hours of dis­tracted brows­ing, or the num­ber of movies /​ TV Series you’ll watch dur­ing the week), again with con­fi­dence in­ter­vals.

Then, by the end of the week, you can check your score, and com­pare it to oth­ers!

(This doesn’t re­quire a ded­i­cated app, it could be a spread­sheet in google docs shared be­tween a few peo­ple, with their email/​con­tacts so they can re­mind each other to fill in the table once the week is over)

• I try to do this by hand, but this would be much more con­ve­nient. Also, in ad­di­tion to us­ing it as a ra­tio­nal­ity train­ing tool I would be in­ter­ested in a pro­gram that col­lects this data and then uses it to di­rectly help me make pre­dic­tions about fu­ture pro­jects.

• I already do this on Pre­dic­tionBook.

• Much to ev­ery­one else’s cha­grin.

I have a bunch of more or less well thought-out ideas of ra­tio­nal­ity games con­cepts, I’ll post a few here. I also did a bit of re­search on what seems teach­able by games and what is not, and talked a bit with Anna about it, but haven’t heard back from her :P

In gen­eral, a big chunk of de­bi­as­ing seems to be about teach­ing eco­nomics and statis­tics (and ap­ply­ing them in ev­ery­day life).

• This is brilli­ant, and has kind of been a dream of mine for a long time.

There is no rea­son why any kind of math­e­mat­ics could not, in prin­ci­ple, be made into a video game.

• Alli­ga­tor eggs is a similar con­cept that could po­ten­tially be used to teach lambda calcu­lus.

• Without ac­tual game lev­els I can’t de­cide whether I’d en­joy Alli­ga­tor eggs or not. In Man­u­fac­to­ria you have to build Tur­ing ma­chines. I doesn’t sound like fun, but I loved it im­mensely.

• Yep, though it may not be worth the trou­ble for some “ad­vanced” maths where the num­ber of peo­ple learn­ing it drops enough for it not to be worth the effort.

Heck, most of the con­tent of classes at school could be bet­ter taught through games.

• First thing that popped into my head:

A farm­ing game that teaches ex­pected out­comes and deal­ing with sunk costs.

Phase 1 is “Make a Plan.” You choose what to plant, what up­grades you want to achieve, etc. If you want to re­in­force the value of plan­ning, other el­e­ments (ex­pand­ing your house, what kind of farm you want to have) should also have you make a plan. Give op­por­tu­ni­ties for peo­ple to pre­pare for risks (storm-proofing, crop di­ver­sity), and if the risk hap­pens, re­mind them that they should have pre­pared.

Phase 2 is where the mind­less click­ing would nor­mally go in a farm­ing game—but now the idea is to re­place some of it with sunk cost type de­ci­sions. You planted corn, but some­one offers you a deal on grape vines that would re­quire los­ing a field of corn. You planned to hire a plant sci­en­tist, but now they’re more ex­pen­sive than they were. Will you go against the plan?

• Has some­thing in com­mon with Agri­cola. Although I think Agri­cola and Seven Won­ders are best at mak­ing you think about op­por­tu­nity costs—there are lots of good things you want to do, and you can’t do them all.

• You planted corn, but some­one offers you a deal on grape vines that would re­quire los­ing a field of corn. You planned to hire a plant sci­en­tist, but now they’re more ex­pen­sive than they were. Will you go against the plan?

If the rules of the game changed once, what is the chance of them chang­ing again? If I de­cide to re­move the field of corn, is there a chance I could later get even bet­ter deal on corn? If the sci­en­tists are more ex­pen­sive that I thought, so I re­place them with work­ers, is there a chance I could later find that work­ers are less effi­cient than ad­ver­tized, so I would had a bet­ter deal with the sci­en­tists?

We should make cer­tain that what the game per­cieves as a sunk cost fal­lacy is re­ally a sunk cost fal­lacy and not some­thing else, for ex­am­ple a ra­tio­nal up­date on the fact that the game some­times changes the rules while play­ing.

One solu­tion is that the game would an­nounce that the change hap­pens ex­actly once per level. It could be em­pha­sised by hav­ing a “SECRET” card, that in the mid­dle of the game turns and re­veals a hid­den rule. (The fact that there are no more “SECRET” cards on the screen should make us feel safe about no more hid­den rules.) The game should not judge player for what they did be­fore the card was shown—per­haps they had some es­ti­mate about the hid­den rule, and already op­ti­mized on this es­ti­mate. But af­ter the rule is shown, the game should re­ward player in how well they played the rest of the game.

For ex­am­ple: You need 5000 cred­its to win the game. After 2000 cred­its a “SECRET” card is played and the ex­ist­ing situ­a­tion is saved. At the end the game shows you the al­ter­na­tive end­ing from the saved point.

• To con­dense what I see as your point: We don’t want to change some­thing and then quickly change it back, or it pun­ishes peo­ple for chang­ing.

But then again, the goal isn’t to teach peo­ple to change—it’s to teach peo­ple to make cor­rect de­ci­sions. If some­thing feels like pun­ish­ment, that’s a game de­sign flaw—you want to make peo­ples’ choices feel in­ter­est­ing, in­formed and im­pact­ful. The real culprit seems to be ei­ther with­hold­ing in­for­ma­tion about changes form the player (could be coun­ter­acted by giv­ing no­tice ahead of time and be­ing clearer about what sorts of things can hap­pen), and mak­ing a sys­tem with lots of cost changes too com­pli­cated (coun­ter­acted by limit­ing the choices pre­sented, break­ing pos­si­ble cost changes into sen­si­ble cat­e­gories, in­tro­duc­ing the player care­fully).

• Per­haps cer­tain vari­ables (like value of corn vs grapes, cost of hiring re­searcher) are ei­ther strictly in­creas­ing or strictly de­creas­ing dur­ing the level, so you can see what is com­ing.

• This is sci­ence rather than ra­tio­nal­ity, but I’d like to see Maxwell’s De­mon the game, a Jezzball-like game where balls are bounc­ing at var­i­ous speeds around a room with a di­vider in the mid­dle. You open & close (or move) a gap in the di­vider to let balls through, with the goal of get­ting the faster-mov­ing balls on one side and the slower balls on the other side. The hot side would get red­der as it “heats up” (the av­er­age ki­netic en­ergy in­creases) while the cold side gets bluer, and the win con­di­tion would be a differ­ence in tem­per­a­ture. More difficult lev­els would have more balls or would re­quire a larger tem­per­a­ture differ­ence.

An even more physics-y ver­sion could in­cor­po­rate the ideal gas law and make it so that the di­vider gets pushed over a lit­tle bit ev­ery time a ball col­lides with it, so that the side of the room with more balls and/​or higher av­er­age speed would tend to ex­pand as the di­vider got pushed away. The win con­di­tion could in­volve both a tem­per­a­ture differ­ence and a lo­ca­tion for the di­vider.

There is already at least one Maxwell’s De­mon game out there, but the one I found isn’t very good (as a game or as physics in­struc­tion). The balls are just red and blue—they don’t vary in speed—and it’s only one level where you have to get 100% sep­a­ra­tion by color.

• Re­lated:

The main point is that video games with re­ally hard lev­els teach the skill of de­liber­ate prac­tice, and pay­ing at­ten­tion to the things you aren’t good at in or­der to get bet­ter.

• 10 Jul 2012 15:07 UTC
1 point

25 un­re­li­able ques­tions. The se­cret-holder may lie once.

• Graph­i­cal cal­ibra­tion game

1) Show the player an image with a bunch of sim­ple images: rec­t­an­gles, smiley heads, cir­cles of var­i­ous color, etc. for about 5 sec­onds.

2) Hide the image, and ask the player to give a 90% con­fi­dence in­ter­val for a value such as the av­er­age size of a figure that he saw.

3) The cor­rect an­swer is then shown, along with the image.

Va­ri­ety can be added by ask­ing for me­dian size, or av­er­age width, or whether there are more red or blue cir­cles, or cor­re­la­tion be­tween width and height, or be­tween size and color (if the cir­cles all vary from light blue to dark blue), or the av­er­age size of red cir­cles, etc.

This al­lows for very tight feed­back loops be­tween guess­ing and see­ing the an­swer, and the game can be re­played pretty much in­finitely.

• ask the player to give a 90% con­fi­dence interval

This could be im­por­tant. Some card games teach cal­ibra­tion, such as Bridge and Spades. (Although it’s not quite the same, be­cause af­ter you guess how many tricks you’ll take, you have some con­trol over it—if you were un­der­con­fi­dent, you can throw tricks away, if you were over­con­fi­dent you can take un­usu­ally large risks.) But they just ask for a sin­gle num­ber, and later you see how close you were but you can’t look back and see how con­fi­dent you were. If you give a con­fi­dence in­ter­val, it’s much eas­ier to see whether you’re well-cal­ibrated.

• Mis­siles keep at­tack­ing your base. You can block each mis­sile by click­ing on the cor­rect shield. Each mis­sile con­tains the words of a fal­la­cious ar­gu­ment and the shield is ei­ther the for­mal name of the fal­lacy be­ing com­mit­ted, a ra­tio­nal counter-ar­gu­ment to the fal­la­cious ar­gu­ment, or the proper rea­son why you should re­ject the fal­lacy.

For ex­am­ple:
Mis­sile “You shouldn’t be a veg­e­tar­ian be­cause Hitler was a veg­e­tar­ian.” Cor­rect shield “Re­v­erse stu­pidity isn’t in­tel­li­gence.” In­cor­rect shield “Hitler lied about so many things that we shouldn’t be­lieve his claims of be­ing a veg­e­tar­ian.” In­cor­rect shield “Ap­plause light”

• More ideas:

-Have users fill out a sur­vey when they first use the game to get a sense of their be­liefs. The game will then pick a mix­ture of as­ser­tions that they likely agree/​dis­agree with.

-Also, have some ‘missles’ be ac­tu­ally valid ar­gu­ments, that give a bonus to what­ever missle base they hit if you don’t mis­tak­enly try to block them with a shield.

• That re­minds me a lit­tle of the Ob­jec­tion! games. I’ve played at least one of them and it was quite fun and ad­dic­tive.

From a re­view:

In Ob­jec­tion! You deal with a mur­der trial. With your vir­tual con­scious­ness at ease you know your client is in­no­cent and have the op­por­tu­nity to defend the pros­e­cu­tor’s line of ques­tion­ing by us­ing one of the 12 ob­jec­tion­able cat­e­gories. The game is not based on mem­o­riza­tion, but on learn­ing to iden­tify and un­der­stand the proper re­sponses. Many ques­tions are also le­gi­t­i­mate and you spec­ify that the ques­tion is proper in or­der to main­tain your silence. After ev­ery ques­tion you can press x or z to get a le­gal ex­pla­na­tion as to the rul­ing of your ob­jec­tion. They also keep a chalk­board re­minder of im­por­tant tips to re­mem­ber. As seen be­low some ques­tions are also worth par­tial or full credit be­cause a va­ri­ety of ob­jec­tions may be suit­able for the situ­a­tion. Rul­ings are reg­u­larly up­dated and it has cor­rect rul­ings for all 50 states, dc and fed­eral court. You also get the op­por­tu­nity to cross ex­am­ine the wit­ness in level 2.

I had a chance to in­ter­view a rep­re­sen­ta­tive from Tran­sMe­dia the com­pany that makes the Ob­jec­tion! se­ries. The game was origi­nally cre­ated for the av­er­age con­sumer, but it quickly be­came de­signed for the pro­fes­sional lawyer. Many lawyers have ex­pressed an in­creased re­sponse time and suc­cess of sharp­en­ing their skills with the game. In fact a Florida judge even or­dered some lawyers to buy the game be­cause they were so in­ad­e­quate at ques­tion­ing.

• One prob­lem for ra­tio­nal­ity games in gen­eral is that an in­cor­rect judg­ment isn’t pun­ished much be­cause you just “start the level over” or some­thing. One solu­tion would be to pe­nal­ize poor de­ci­sions more heav­ily, e.g. by mak­ing peo­ple start fur­ther back in the game than would be ex­pected when, say, miss­ing a jump in a plat­former.

• One prob­lem for ra­tio­nal­ity games in gen­eral is that an in­cor­rect judg­ment isn’t pun­ished much be­cause you just “start the level over” or some­thing.

Is that re­ally a prob­lem? It’s my un­der­stand­ing that get­ting rapid feed­back, and an op­por­tu­nity to retry the failed at­tempt while your pre­vi­ous failure is still fresh in your mem­ory, is much more use­ful for skill ac­qui­si­tion than hav­ing each failure be max­i­mally frus­trat­ing.

• Hmm… per­haps you could just tighten the time/​# of moves limit? To give fast, ap­pro­pri­ate up­dat­ing an ad­van­tage over a brute-force ap­proach?

The prob­lem would need to change a bit each time you tried it, so that you had to learn within the round.

Zoom­bi­nis ac­tu­ally had some great ex­am­ples of this, where you had to learn what test was be­ing ap­plied, and how to pass the test, by in­duc­tion. The ex­act crite­ria were ran­domly de­ter­mined, so you had to solve the prob­lem anew each time. And you were pe­nal­ized for tak­ing too long to learn, so be­ing good at the game in­volved com­ing up with an effi­cient learn­ing al­gorithm, rather than just an ad­e­quate one.

• Oh man, I played Zoom­bi­nis when I was a kid, I loved that game and haven’t thought about it in for­ever.

• Zoom­bi­nis pro tip: The game al­lows you to cre­ate up to two zoom­bi­nis of each hair/​eye/​nose/​trans­port con­figu­ra­tion. You can make things way eas­ier by mak­ing your party con­sist of 8 twinned zoom­bini pairs as op­posed to the usual 16 dis­tinct zoom­bi­nis. (For puz­zles that de­pend on zoom­bini fea­tures, which is pretty much all of them, the solu­tion for a given zoom­bini and it’s twin will be the same.)

I love how the game’s wikipe­dia page has a fairly de­tailed ex­pla­na­tion of ev­ery puz­zle...

• So did I!

And, as far as I can re­mem­ber, not only were the puz­zles well de­signed, but there was a rea­son­ably good over­ar­ch­ing story with a definite goal, which meant the puz­zles had pur­pose and there was a lit­tle sense of ad­ven­ture too.

(I should re­ally find the disk and play it again… hehe.)

• Another com­mon solu­tion is to ran­domly gen­er­ate prob­lems so that you can never learn “by rote” how to pass the level.

They can still “grind” by try­ing ran­dom things un­til they hap­pen to suc­ceed by chance, but the wider con­text of the game can dis­cour­age that (for ex­am­ple by just show­ing how many tries it took you)

• Not sure how well it fits, but this was an awe­somely done game for teach­ing di­vi­sion.

http://​​games.cs.wash­ing­ton.edu/​​Refrac­tion/​​

• Plan­ning calibration

The ba­sis is an An­gry-Birds-like game, where have a stock of 10 can­non­balls, which you shoot to knock down stuff (with ded­i­cated tar­gets, blocks with var­i­ous shapes and prop­er­ties, etc.).

BUT, be­fore you start play­ing, you must es­ti­mate how many can­non­balls it will take you to pass this par­tic­u­lar level. At the end of the level, you score 2 points for each re­main­ing can­non­ball, minus one for each can­non­ball you planned but didn’t shoot, or shot but didn’t plan.

The gen­eral idea is to help face the plan­ning fal­lacy and cal­ibrate ac­cord­ingly; the same idea can be ap­plied to all kinds of games (on a plat­former: how many lives will it take you to pass this level, etc.).

• Would have to be ran­domly gen­er­ated lev­els with no restart so that play­ers can’t just set their es­ti­mate low and play un­til they achieve that es­ti­mate.

• Another well-re­garded ed­u­ca­tional app is Math Evolve.

• That kind of game is rel­a­tively easy, be­cause the prob­lems are triv­ial (“2 + 6 = ?”); the task is only to pre­sent them in an in­ter­est­ing way. You could do a World of War­craft clone where you travel across a 3D map and peo­ple give you quests in form of “2 + 6 = ?” and then you have to fight a dragon with “8” writ­ten on his belly (there are also other drag­ons available, but those will kill you). There is no re­la­tion be­tween a dragon and a num­ber 8, there­fore you have com­plete free­dom in de­sign­ing such games.

The only draw­back is that with­out care, this could de­gen­er­ate into “guess­ing the teacher’s pass­word”, be­cause the un­der­ly­ing model is just ques­tion/​an­swer with­out a mechanism ex­plain­ing why the given an­swer is cor­rect. You could re­place texts “2 + 6 = ?” and “8” with texts “Which re­li­gion is the true one?” and “Chris­ti­an­ity”, and maybe you wouldn’t even need to re­com­pile the bi­na­ries. -- OK, this is ex­ag­ger­ated, be­cause with math the game can gen­er­ate many new ques­tion/​an­swer pairs in­stead of hav­ing a fixed database of them, so it is eas­ier to learn some al­gorithm to de­ter­mine the cor­rect an­swer in­stead of mem­o­riz­ing them, which is ex­actly what we want to do. But the point is that this game does not show you why for a ques­tion “2 + 6 = ?” the an­swer “8” is more cor­rect than “9″. It just re­wards the former and pun­ishes the lat­ter. (Could be fixed by adding an an­i­ma­tion that dis­plays 2 red and 6 yel­low spheres on one side, 9 green spheres on other side, then moves them to pairs and shows that on one side there is more.)

• Skill: Fungibility

Make a Phoenix-like shooter game with differ­ent kinds of am­mu­ni­tion that do differ­ent amounts of dam­age, and can be changed into each other at some (chang­ing?) ra­tio. Give the play­ers a limited amount of am­mu­ni­tion that can be re­plen­ished by kil­ling en­emy ships, along with oc­ca­sional pack­ets of ad­di­tional ammo.

Have the en­emy ships have differ­ent amounts of health, then con­trive their or­der so that the player does sig­nifi­cantly bet­ter by suc­cess­fully get­ting the max­i­mum amount of dam­age, given the am­mu­ni­tion they cur­rently have.

The game should be rigged so that you run out of ammo and need to dodge things for a while if you don’t do the ar­bi­trage, and so that you have a lit­tle ex­tra left over if you do.

I prob­a­bly made this a lit­tle too com­pli­cated some­where.

Thoughts?

It seems like a challenge of do­ing this is that a lot of ra­tio­nal­ity train­ing is about en­courag­ing peo­ple to use differ­ent men­tal pro­cesses to ac­com­plish their goals than the ones they nor­mally use. In a game, the tasks are of­ten straight­for­ward in a away that seems to cut out a lot of ra­tio­nal­ity.

Like, whether or not I’m be­ing spe­cific or ap­ply­ing fun­gi­bil­ity is difficult to mea­sure, if all you have is my ac­tions in An­gry Birds. Whether or not I’m fram­ing prob­lems well (kill the pigs vs. get this tac­ti­cal im­ple­men­ta­tion of knock­ing down the blocks) might be no­tice­able based on the similar­ity of my moves, but it would be tricky to no­tice and re­ward that.

More com­pli­cated games like Civ­i­liza­tion could prob­a­bly train some ra­tio­nal­ity skills, but it’s also to­tally easy to just get stuck in the game. Also, they tend to be waaay longer than I’d like.

It seems like a ba­sic is­sue here is how to make it so that you need to di­rectly use a ra­tio­nal­ity sub­skill in or­der to play the game well.