After some more thought, I realized that making a game with fancy graphics and complex gameplay would probably not be a good idea for a first project to try.
A better idea would be a simple text-based game you play in your browser, probably running on either PHP or Python.
This might not have as much fun appeal as a traditional video game, since it would probably look like a university exam, but it could still be an effective way for us to measure our rationality.
So far I thought of the following ideas:
1) Multiple Choice questions, with only one right answer. Not just a quiz of trivia you memorized from Less Wrong, but applications of the techniques taught here. Also, the standard tests that reveal if you’ve learned how to not fall for the standard biases.
2) Questions that require you to do some math, and enter your answer, which can be scored on how close it is to the correct answer.
3) A “spot the logical fallacy” game, or a “spot the bias” game. The player is presented with a few paragraphs of text. It could be a news article, a section of a short story, a conversation between two people, or maybe something else. The purpose is to spot the first sentence that contains a logical fallacy, or a bias. Once you select which sentence contains the error, you have to say what type of error it is. Maybe typing in the name of the fallacy, or maybe selecting it from a list of the fallacies that have pages about them on the LW Wiki.
We could also implement questions that start with an introduction that you can read before the timer starts ticking, then you proceed to the timed part, where you are scored based on how quickly you answer. Maybe some questions could have a time limit of a few seconds, to force an immediate response.
This LW game could be integrated with the rest of the LW website, where your score is linked to your username. Though this feature is entirely optional. The wiki already has a separate user account not linked directly to your main LW user account, so the game could do the same. The game would keep track of which questions you got right, and how long it took you to answer. This would give speedreaders an advantage, but maybe this isn’t a problem, because speedreading is a generally useful skill to learn.
The game should keep track of which questions you’ve already answered, so it doesn’t ask you the same question again. Or maybe for the questions where you need to do math, it could ask you again with randomized numbers. Maybe there could be a calculator built into the question webpage, with functions for calculating probabilities, etc.
There should also be a way to compare your scores with other players, or see how you rank.
To generate questions for the game, we could use a comments thread in the main LW site, or we could use the LW wiki, or we could set up a special submission form in the game itself.
another random idea: If you give an answer that you think is right, but the game says is wrong, there should be a way to enter your explanation for why your answer is right. Someone would then check these messages, and if your explanation is valid, the game could award you double points.
(Another random idea I had was to make a text adventure game, where you participate in conversations, and sometimes need to interrupt a conversation to point out a logical fallacy, to prevent the conversation from going off-track and preventing you from getting the information you needed from the conversation. But that would be more of a challenge to implement, and generate content for.)
3) A “spot the logical fallacy” game, or a “spot the bias” game. The player is presented with a few paragraphs of text. It could be a news article, a section of a short story, a conversation between two people, or maybe something else. The purpose is to spot the first sentence that contains a logical fallacy, or a bias. Once you select which sentence contains the error, you have to say what type of error it is. Maybe typing in the name of the fallacy, or maybe selecting it from a list of the fallacies that have pages about them on the LW Wiki.
I think the problem is that we lack a good game mechanic. Come up with a mechanic, and the goals can follow, but it’s hard to go from a goal like ‘calibrate yourself to avoid overconfidence’ to a fun game. We need to think about how to borrow games like Zendo and repurpose them.
That LogicTutor site you linked to provides a good, basic introduction to a few concepts and fallacies. However, the practice problems just ask you to identify which fallacy is in the sentence they give you. They’re missing the other half of the game, which is spotting the fallacy in a block of text that’s deliberately designed to hide the fallacy. I’ll keep looking in case someone has already made a game that contains this part.
One way to make the game more fun would be to have interesting text to find the fallacy in. Eliezer’s short stories are a good example of this. Though for the purpose of the game, we would need just short segments of stories, which contain one clear example of a fallacy. Preferably one that’s well-hidden, but obvious once you see it. Also, to keep players on their toes, we could include segments that don’t actually contain a fallacy, and players would have the option of saying that there is no fallacy.
And as I mentioned before, another idea is to flesh out the stories even more, so that it could be expanded into a mystery game, or an adventure game, or an escape-the-room game, where in order to continue you need to talk to people, and some of these people will give inaccurate information, because they didn’t notice a flaw in their own reasoning, and you will need to point out the flaw in their reasoning before they will give you the accurate information.
You would also have to choose your replies during the conversation, and have to choose a reply that doesn’t introduce a new fallacy and send the conversation off in the wrong direction. Many of the possible responses would be to question why the person believes specific things that they just said. Maybe there could also be a feature where you could interrupt the person in the middle of what they’re saying, to point out the problem. Optionally, score the player based on how long they took, and how many wrong paths they went down before finding the correct path.
And if this still doesn’t make the game fun, then there are ways to make people want to play games even if they aren’t fun.
One way is to make the not-so-fun game into a small but necessary part of a bigger game, that is fun.
Another way is to make it into a Flash game, and submit it to a site that tracks your achievements in the games, and gives you an overall score over all the games you’ve played. Examples of sites like this are Kongregate.com, and Newgrounds.com. If you’re lucky, the game might even get to spend a few days on the site’s front page, or get a limited-time challenge, for extra bonus points.
If the game could be made into a series, that would be even better.
So far I’ve only seen one “educational” game get promoted to the front page of Kongragate.com, with its own limited-time-only challenge. That was Globetrotter XL. The game says the name of a city, and you have to click on where that city is, on a map of the world with no border lines or labels. You’re scored for how close you click to the actual city. Unfortunately, I ended up not completing this game’s limited-time challenge, because the game was so unforgiving, and my knowledge of geography was so poor.
Anyway, while it would be a really nice bonus if the spot-the-fallacy game became popular outside LW, the main purpose of the game is to give current LW members an objective way to test their rationality, even if the game isn’t especially fun.
There have been several LW posts now talking about how desperately we need a way to measure our rationality, and so far I haven’t seen any serious proposals that anyone is actually working on. Or maybe there’s a project already started, and I just didn’t notice it because I got so far behind on reading the LW posts.
This game/quiz/whatever is a proposal for a way to go from having no objective way to measure our rationality, to at least having something.
And as I mentioned before, another idea is to flesh out the stories even more, so that it could be expanded into a mystery game, or an adventure game, or an escape-the-room game, where in order to continue you need to talk to people, and some of these people will give inaccurate information, because they didn’t notice a flaw in their own reasoning, and you will need to point out the flaw in their reasoning before they will give you the accurate information.
Many of the possible responses would be to question why the person believes specific things that they just said.
(Starting to think maybe we could use a wiki page, even if only for links and ideas. This game discussion is now spread out over something like 5 LW articles...)
I’m going to use this open thread to once again suggest the idea of a Less Wrong video game.
( Here’s a link to the post I made last month about it )
After some more thought, I realized that making a game with fancy graphics and complex gameplay would probably not be a good idea for a first project to try.
A better idea would be a simple text-based game you play in your browser, probably running on either PHP or Python.
This might not have as much fun appeal as a traditional video game, since it would probably look like a university exam, but it could still be an effective way for us to measure our rationality.
So far I thought of the following ideas:
1) Multiple Choice questions, with only one right answer. Not just a quiz of trivia you memorized from Less Wrong, but applications of the techniques taught here. Also, the standard tests that reveal if you’ve learned how to not fall for the standard biases.
2) Questions that require you to do some math, and enter your answer, which can be scored on how close it is to the correct answer.
3) A “spot the logical fallacy” game, or a “spot the bias” game. The player is presented with a few paragraphs of text. It could be a news article, a section of a short story, a conversation between two people, or maybe something else. The purpose is to spot the first sentence that contains a logical fallacy, or a bias. Once you select which sentence contains the error, you have to say what type of error it is. Maybe typing in the name of the fallacy, or maybe selecting it from a list of the fallacies that have pages about them on the LW Wiki.
We could also implement questions that start with an introduction that you can read before the timer starts ticking, then you proceed to the timed part, where you are scored based on how quickly you answer. Maybe some questions could have a time limit of a few seconds, to force an immediate response.
This LW game could be integrated with the rest of the LW website, where your score is linked to your username. Though this feature is entirely optional. The wiki already has a separate user account not linked directly to your main LW user account, so the game could do the same. The game would keep track of which questions you got right, and how long it took you to answer. This would give speedreaders an advantage, but maybe this isn’t a problem, because speedreading is a generally useful skill to learn.
The game should keep track of which questions you’ve already answered, so it doesn’t ask you the same question again. Or maybe for the questions where you need to do math, it could ask you again with randomized numbers. Maybe there could be a calculator built into the question webpage, with functions for calculating probabilities, etc.
There should also be a way to compare your scores with other players, or see how you rank.
To generate questions for the game, we could use a comments thread in the main LW site, or we could use the LW wiki, or we could set up a special submission form in the game itself.
another random idea: If you give an answer that you think is right, but the game says is wrong, there should be a way to enter your explanation for why your answer is right. Someone would then check these messages, and if your explanation is valid, the game could award you double points.
(Another random idea I had was to make a text adventure game, where you participate in conversations, and sometimes need to interrupt a conversation to point out a logical fallacy, to prevent the conversation from going off-track and preventing you from getting the information you needed from the conversation. But that would be more of a challenge to implement, and generate content for.)
This could be used to make a game based off of Dungeons and Discourse.
When you attack, you have to select an argument without a flaw, or it gets blocked. When the opponent attacks, if you find a flaw, it deals no damage.
This has already been done many times as part of critical thinking courses; people don’t use free sites like http://www.wwnorton.com/college/phil/logic3/ because they’re boring and hard.
I think the problem is that we lack a good game mechanic. Come up with a mechanic, and the goals can follow, but it’s hard to go from a goal like ‘calibrate yourself to avoid overconfidence’ to a fun game. We need to think about how to borrow games like Zendo and repurpose them.
Thanks for the link.
That LogicTutor site you linked to provides a good, basic introduction to a few concepts and fallacies. However, the practice problems just ask you to identify which fallacy is in the sentence they give you. They’re missing the other half of the game, which is spotting the fallacy in a block of text that’s deliberately designed to hide the fallacy. I’ll keep looking in case someone has already made a game that contains this part.
One way to make the game more fun would be to have interesting text to find the fallacy in. Eliezer’s short stories are a good example of this. Though for the purpose of the game, we would need just short segments of stories, which contain one clear example of a fallacy. Preferably one that’s well-hidden, but obvious once you see it. Also, to keep players on their toes, we could include segments that don’t actually contain a fallacy, and players would have the option of saying that there is no fallacy.
And as I mentioned before, another idea is to flesh out the stories even more, so that it could be expanded into a mystery game, or an adventure game, or an escape-the-room game, where in order to continue you need to talk to people, and some of these people will give inaccurate information, because they didn’t notice a flaw in their own reasoning, and you will need to point out the flaw in their reasoning before they will give you the accurate information.
You would also have to choose your replies during the conversation, and have to choose a reply that doesn’t introduce a new fallacy and send the conversation off in the wrong direction. Many of the possible responses would be to question why the person believes specific things that they just said. Maybe there could also be a feature where you could interrupt the person in the middle of what they’re saying, to point out the problem. Optionally, score the player based on how long they took, and how many wrong paths they went down before finding the correct path.
And if this still doesn’t make the game fun, then there are ways to make people want to play games even if they aren’t fun.
One way is to make the not-so-fun game into a small but necessary part of a bigger game, that is fun.
Another way is to make it into a Flash game, and submit it to a site that tracks your achievements in the games, and gives you an overall score over all the games you’ve played. Examples of sites like this are Kongregate.com, and Newgrounds.com. If you’re lucky, the game might even get to spend a few days on the site’s front page, or get a limited-time challenge, for extra bonus points.
If the game could be made into a series, that would be even better.
So far I’ve only seen one “educational” game get promoted to the front page of Kongragate.com, with its own limited-time-only challenge. That was Globetrotter XL. The game says the name of a city, and you have to click on where that city is, on a map of the world with no border lines or labels. You’re scored for how close you click to the actual city. Unfortunately, I ended up not completing this game’s limited-time challenge, because the game was so unforgiving, and my knowledge of geography was so poor.
Anyway, while it would be a really nice bonus if the spot-the-fallacy game became popular outside LW, the main purpose of the game is to give current LW members an objective way to test their rationality, even if the game isn’t especially fun.
There have been several LW posts now talking about how desperately we need a way to measure our rationality, and so far I haven’t seen any serious proposals that anyone is actually working on. Or maybe there’s a project already started, and I just didn’t notice it because I got so far behind on reading the LW posts.
This game/quiz/whatever is a proposal for a way to go from having no objective way to measure our rationality, to at least having something.
Well that would be an interesting game mechanic.
How about a courtroom drama?
I object!
(Alternate Higurashi ending: /me claws throat open to get at insects on me)
Incidentally, another link right up your alley: http://projects.csail.mit.edu/worldview/about
(Starting to think maybe we could use a wiki page, even if only for links and ideas. This game discussion is now spread out over something like 5 LW articles...)
5 articles? I only know of 2 articles where this topic was discussed. Can you post a link to the other 3, please?
Anyway, I went ahead and created a wiki page for discussing the game idea, and posted links to the two threads I know of:
http://wiki.lesswrong.com/wiki/The_Less_Wrong_Video_Game
It certainly feels like it’s been spread out over more than 2 open threads!