“Basic decision theory points” feels like a pretty weird description of something that even quite smart people still frequently disagree on, has no formal description, and indeed often turns out to be the crux of an argument.
I currently don’t think it’s worth my time figuring things out here much more, but that’s mostly because I do have some reasonable confidence that thinking about decision theory harder probably won’t produce any quick breakthroughs. But if I was in the world that MIRI faced 15 years ago, my guess is I would have thought it was worth investing in quite a bit, in case it does turn out to be relatively straightforward (which it so far has not turned out to be).
rather than coming up with decision theories that work well in limits that aren’t important for the most important kinds of AI futurism
Pushing more straightforwardly back on this: I do not think our current understanding of decision-theory is better in the mundane case than the limit case. Of course the reason to look at limiting cases is because you always do that in math because the limiting cases often turn out easier, not harder than the mundane case.
“Basic decision theory points” feels like a pretty weird description of something that even quite smart people still frequently disagree on, has no formal description, and indeed often turns out to be the crux of an argument.
I currently don’t think it’s worth my time figuring things out here much more, but that’s mostly because I do have some reasonable confidence that thinking about decision theory harder probably won’t produce any quick breakthroughs. But if I was in the world that MIRI faced 15 years ago, my guess is I would have thought it was worth investing in quite a bit, in case it does turn out to be relatively straightforward (which it so far has not turned out to be).
Pushing more straightforwardly back on this: I do not think our current understanding of decision-theory is better in the mundane case than the limit case. Of course the reason to look at limiting cases is because you always do that in math because the limiting cases often turn out easier, not harder than the mundane case.