As far as I can tell, LW was created explicitly with the goal of producing rationalists, one desirable side effect of which was the creation of friendly AI researchers. Decision theory plays a prominent role in Eliezer’s conception of friendly AI, since a decision theory is how the AI is supposed to figure out the right thing to do. The obvious guesses don’t work in the presence of things like other agents that can read the AI’s source code, so we need to find some non-obvious guesses because that’s something that could actually happen.
Hey, I think your tone here comes across as condescending, which goes against the spirit of a ‘stupid questions’ thread, by causing people to believe they will lose status by posting in here.
This was what I gathered from reading the beginning of the TDT paper: “There’s this one decision theory that works in every single circumstance except for this one crazy sci-fi scenario that might not even be physically possible, and then there’s this other decision theory that works in said sci-fi scenario but not really anywhere else. We need to find a decision theory that combines these two in order to always work, including in this one particular sci-fi scenario.”
I guess it might be useful for AI research, but I don’t see why I would need to learn it.
the sci-fi bit is only to make it easier to think about. The real world scenarios it corresponds to require the reader to have quite a bit more background material under their belt to reason carefully about.
What are the real world scenarios it corresponds to? The only one I know of is the hitchhiker one, which is still pretty fantastic. I’m interested in learning about this.
It’s not obvious to me how tragedy of the commons/prisoner’s dilemma is isomorphic to Newcomb’s problem, but I definitely believe you that it could be. If TDT does in fact present a coherent solution to these types of problems, then I can easily see how it would be useful. I might try to read the pdf again sometime. Thanks.
The example I learned involved cutting across the grass as a shortcut instead of walking on a path. No one person can damage the grass, but if everyone walks across the grass, it dies, reducing everyone’s utility more than gained by the shortcut.
For a real world example, I suspect that one’s intuition about the acceptability of copyright piracy depends on one’s intuitions about committing to pay for content and the amount of content that would exist.
In other words, it seems intuitive that the truly rational would voluntarily co-operate to avoid tragedies of the commons. But voluntary commitment to a course of action is hard to formally justify.
It is done for AI research. The “real world scenarios” usually involve several powerful AIs, so depending on what you mean by “sci-fi” they might not apply. (Even if you don’t consider AIs sci-fi, the usual problem statements make lots of simplifying assumptions that are not necessarily realistic, like perfect guessing and things like that, but that’s just like ignoring friction in physics problems, nobody expects for the exact same thing to happen in practice.)
Why is everyone so intereted in decision theory? Especially the increasingly convoluted variants with strange acronyms that seem to be popping up
As far as I can tell, LW was created explicitly with the goal of producing rationalists, one desirable side effect of which was the creation of friendly AI researchers. Decision theory plays a prominent role in Eliezer’s conception of friendly AI, since a decision theory is how the AI is supposed to figure out the right thing to do. The obvious guesses don’t work in the presence of things like other agents that can read the AI’s source code, so we need to find some non-obvious guesses because that’s something that could actually happen.
Hey, I think your tone here comes across as condescending, which goes against the spirit of a ‘stupid questions’ thread, by causing people to believe they will lose status by posting in here.
Fair point. My apologies. Getting rid of the first sentence.
Thanks!
data point: I didn’t parse it as condescending at all.
Did you read it before it was rephrased?
Ah, I see there was a race condition. I’ll retract my comment.
This was what I gathered from reading the beginning of the TDT paper: “There’s this one decision theory that works in every single circumstance except for this one crazy sci-fi scenario that might not even be physically possible, and then there’s this other decision theory that works in said sci-fi scenario but not really anywhere else. We need to find a decision theory that combines these two in order to always work, including in this one particular sci-fi scenario.”
I guess it might be useful for AI research, but I don’t see why I would need to learn it.
the sci-fi bit is only to make it easier to think about. The real world scenarios it corresponds to require the reader to have quite a bit more background material under their belt to reason carefully about.
What are the real world scenarios it corresponds to? The only one I know of is the hitchhiker one, which is still pretty fantastic. I’m interested in learning about this.
Any kind of tragedy of the commons type scenario would qualify.
It’s not obvious to me how tragedy of the commons/prisoner’s dilemma is isomorphic to Newcomb’s problem, but I definitely believe you that it could be. If TDT does in fact present a coherent solution to these types of problems, then I can easily see how it would be useful. I might try to read the pdf again sometime. Thanks.
They aren’t isomorphic problems, however it is the case that CDT two-boxes and defects while TDT one boxes and co-operates (against some opponents).
In general, there are situations where act utilitarianism says a choice is permissible, but rule utilitarianism says the choice is not permissible.
The example I learned involved cutting across the grass as a shortcut instead of walking on a path. No one person can damage the grass, but if everyone walks across the grass, it dies, reducing everyone’s utility more than gained by the shortcut.
For a real world example, I suspect that one’s intuition about the acceptability of copyright piracy depends on one’s intuitions about committing to pay for content and the amount of content that would exist.
In other words, it seems intuitive that the truly rational would voluntarily co-operate to avoid tragedies of the commons. But voluntary commitment to a course of action is hard to formally justify.
If everyone walks across the grass instead of on the path, that is strong evidence that the path is in the wrong place.
It does not follow from this that it would be good for everyone to cut across the grass.
http://lesswrong.com/lw/4sh/how_i_lost_100_pounds_using_tdt/
I imagine at least half of those upvotes were generated by the title alone.
It is done for AI research. The “real world scenarios” usually involve several powerful AIs, so depending on what you mean by “sci-fi” they might not apply. (Even if you don’t consider AIs sci-fi, the usual problem statements make lots of simplifying assumptions that are not necessarily realistic, like perfect guessing and things like that, but that’s just like ignoring friction in physics problems, nobody expects for the exact same thing to happen in practice.)