If I understand the research program under discussion, certain ideas are answered “somebody else will”. e.g.
Don’t build RSI, build AI with limited improvement capabilities (like humans) and use Moore’s law to get speedup. “but somebody else will”
Build it so that all it does is access a local store of data (say a cache of the internet) and answer multiple choice questions (or some other limited function). Don’t build it to act. “but somebody else will”
etc. every safety suggestion can be met with “somebody else will build an AI that does not have this safety feature”.
So: make it Friendly. “but somebody else won’t”.
This implies: make it Friendly and help it take over the world to a sufficient degree that “somebody else” has no opportunity to build non-Friendly AI.
I think it is hugely unlikely that intelligence of the level being imagined is possible in anything like the near future, and “recursive self improvement” is very likely to be a lot more limited than projected (there’s a limit to how much code can be optimized, P!=NP which severely bounds general search optimization, there’s only so much you can do with “probably true” priors, and the physical world itself is too fuzzy to support much intellegent manipulation). But I could be wrong.
So, if you guys are planning to take over the world with your Friendly AI, I hope you get it right. I’m surprised there isn’t an “Open Friendliness Project” to help answer all the objections and puzzles that commenters on this thread.
If Friendliness has already been solved, I’m reminded of Dr. Strangelove: it does no good to keep it a secret!
If it isn’t, is it moral to work on more dangerous aspects (like reflectivity) without Friendliness worked out beforehand?
Eliezer taught you rationality, so figure it out!
If I understand the research program under discussion, certain ideas are answered “somebody else will”. e.g.
Don’t build RSI, build AI with limited improvement capabilities (like humans) and use Moore’s law to get speedup. “but somebody else will”
Build it so that all it does is access a local store of data (say a cache of the internet) and answer multiple choice questions (or some other limited function). Don’t build it to act. “but somebody else will”
etc. every safety suggestion can be met with “somebody else will build an AI that does not have this safety feature”.
So: make it Friendly. “but somebody else won’t”.
This implies: make it Friendly and help it take over the world to a sufficient degree that “somebody else” has no opportunity to build non-Friendly AI.
I think it is hugely unlikely that intelligence of the level being imagined is possible in anything like the near future, and “recursive self improvement” is very likely to be a lot more limited than projected (there’s a limit to how much code can be optimized, P!=NP which severely bounds general search optimization, there’s only so much you can do with “probably true” priors, and the physical world itself is too fuzzy to support much intellegent manipulation). But I could be wrong.
So, if you guys are planning to take over the world with your Friendly AI, I hope you get it right. I’m surprised there isn’t an “Open Friendliness Project” to help answer all the objections and puzzles that commenters on this thread.
If Friendliness has already been solved, I’m reminded of Dr. Strangelove: it does no good to keep it a secret!
If it isn’t, is it moral to work on more dangerous aspects (like reflectivity) without Friendliness worked out beforehand?