Since this seems like a pretty transparent metaphor for Friendly AI, it looks as though Eliezer is planning to go through with his idea of crowdsourcing FAI research. Any predictions for how this is going to go? I’m personally not optimistic that the subreddit is actually going to produce any important, novel results*, but at the very least, it’ll increase exposure to the idea of FAI with a general audience. (After all, HPMoR was what originally brought me to LW.)
* It seems to me that the main strength of crowdsourcing in solving problems is the ability to propose a truly gigantic amount of solutions in a very short amount of time, which only helps if (a) the true solution is easy enough to guess that someone can stumble upon it largely by chance, (b) other people then recognize the solution as a good one and upvote it, and (c) the solution is easily testable to see if it is a good one or a bad one (otherwise people will keep on proposing solutions without realizing that they’ve already stumbled across the right answer). All three of these were true of HPMoR; all of them are probably false in the context of FAI research.
One of the main things that stops me from writing about things—FAI included—is that if something feels very important, anxiety kicks in and inhibits the thought-to-keyboard process. If that problem is at all common, then a thin veil of frivolity will do wonders for research productivity.
That seems fair, but I’d say that unless you’re already intelligent enough to do important, original work in the field of FAI (or any other field of mathematics, really), a productivity boost won’t help much. To use an analogy: a car whose engine is broken won’t run no matter how much gasoline you put in its tank.
(Not to imply that the people who frequent /r/hpmor are unintelligent, just that the bar for doing successful FAI research is really, really high, and unless you can clear that bar, increasing the number of people working on the problem isn’t likely to help—in my view, anyway. I could be wrong.)
I quite liked the story idea until I realised that its a pretty transparent metaphor for Friendly AI… no, wait, it actually is a story about FAI. Starting off with worldbuilding a fantasy magicpunk setting and then suddenly switching to FAI seems… kinda like bait and switch?
Having said that, I really like this setting. The main problem is that there seem to be two entirely different themes—FAI and sorcerers taking over the world. If you start discussing hard maths you are going to lose many readers, but then if the goal is to inspire FAI work, does this matter?
The secondary problem is if you can just reset the timeline as many times as you want, there is no sense of urgency or tension. Maybe they discover that each time they reset, cracks start to appear in the walls between realms , deamon summoning becomes easier, and the daemons are one step closer to being able to break through on their own?
Meta: is there any point in discussing this here when the reddit conversation is so much bigger? I’m probably just going to copy my comment over.
Since this seems like a pretty transparent metaphor for Friendly AI, it looks as though Eliezer is planning to go through with his idea of crowdsourcing FAI research. Any predictions for how this is going to go? I’m personally not optimistic that the subreddit is actually going to produce any important, novel results*, but at the very least, it’ll increase exposure to the idea of FAI with a general audience. (After all, HPMoR was what originally brought me to LW.)
* It seems to me that the main strength of crowdsourcing in solving problems is the ability to propose a truly gigantic amount of solutions in a very short amount of time, which only helps if (a) the true solution is easy enough to guess that someone can stumble upon it largely by chance, (b) other people then recognize the solution as a good one and upvote it, and (c) the solution is easily testable to see if it is a good one or a bad one (otherwise people will keep on proposing solutions without realizing that they’ve already stumbled across the right answer). All three of these were true of HPMoR; all of them are probably false in the context of FAI research.
One of the main things that stops me from writing about things—FAI included—is that if something feels very important, anxiety kicks in and inhibits the thought-to-keyboard process. If that problem is at all common, then a thin veil of frivolity will do wonders for research productivity.
That seems fair, but I’d say that unless you’re already intelligent enough to do important, original work in the field of FAI (or any other field of mathematics, really), a productivity boost won’t help much. To use an analogy: a car whose engine is broken won’t run no matter how much gasoline you put in its tank.
(Not to imply that the people who frequent /r/hpmor are unintelligent, just that the bar for doing successful FAI research is really, really high, and unless you can clear that bar, increasing the number of people working on the problem isn’t likely to help—in my view, anyway. I could be wrong.)
I quite liked the story idea until I realised that its a pretty transparent metaphor for Friendly AI… no, wait, it actually is a story about FAI. Starting off with worldbuilding a fantasy magicpunk setting and then suddenly switching to FAI seems… kinda like bait and switch?
Having said that, I really like this setting. The main problem is that there seem to be two entirely different themes—FAI and sorcerers taking over the world. If you start discussing hard maths you are going to lose many readers, but then if the goal is to inspire FAI work, does this matter?
The secondary problem is if you can just reset the timeline as many times as you want, there is no sense of urgency or tension. Maybe they discover that each time they reset, cracks start to appear in the walls between realms , deamon summoning becomes easier, and the daemons are one step closer to being able to break through on their own?
Meta: is there any point in discussing this here when the reddit conversation is so much bigger? I’m probably just going to copy my comment over.