Yeah I think the ‘doing two things at once’ is an issue, though my main intended audience for this paper is academic philosophers and decision theorists who are—as a rule—both mathy and new to AI safety stuff.
Your other points sound kinda like a ‘Theorems are slow and detailed’ complaint to which I say: yes, but the detail helps guide our search for solutions. For example, it was thinking about Theorems 2 and 3-ish stuff that first got me thinking that incomplete preferences might help with shutdownability.
I am convinced of Independence as a requirement of rationality, for paying-to-avoid-information and money-pump reasons (like Yudkowsky’s), plus I think the Allais preferences aren’t much evidence against. I went for Indifference Between Indifference-Shifted Lotteries because it’s a little weaker (though since writing the paper I’ve been convinced that it’s not significantly weaker. Basically every endorsed decision theory that violates Independence also violates IBISL).
I do not generally have a problem with slow and detailed theorems. I seem to have tastes different from yours about when theorems add understanding. But if it got you thinking about incompleteness, then it’s done its job.
Yeah I think the ‘doing two things at once’ is an issue, though my main intended audience for this paper is academic philosophers and decision theorists who are—as a rule—both mathy and new to AI safety stuff.
Your other points sound kinda like a ‘Theorems are slow and detailed’ complaint to which I say: yes, but the detail helps guide our search for solutions. For example, it was thinking about Theorems 2 and 3-ish stuff that first got me thinking that incomplete preferences might help with shutdownability.
I am convinced of Independence as a requirement of rationality, for paying-to-avoid-information and money-pump reasons (like Yudkowsky’s), plus I think the Allais preferences aren’t much evidence against. I went for Indifference Between Indifference-Shifted Lotteries because it’s a little weaker (though since writing the paper I’ve been convinced that it’s not significantly weaker. Basically every endorsed decision theory that violates Independence also violates IBISL).
I do not generally have a problem with slow and detailed theorems. I seem to have tastes different from yours about when theorems add understanding. But if it got you thinking about incompleteness, then it’s done its job.