the development of a new ‘mental martial art’ of systematically correct reasoning
Unpopular opinion: Rationality is less about martial arts moves than about adopting an attitude of intellectual good faith and consistently valuing impartial truth-seeking above everything else that usually influences belief selection. Motivating people (including oneself) to adopt such an attitude can be tricky, but the attitude itself is simple. Inventing new techniques is good but not necessary.
What does it mean for a bit to pop into existence? As I see it, if I measure a particle’s spin at time t, then it’s either timelessly the case that the result is “up” or timelessly the case that the result is “down”. Maybe this is an issue of A Theory versus B Theory?
(I agree with this and made the uncle comment before seeing it. Also, my experience wasn’t like that most of the time; I think it was mainly that way toward the end of LW 1.0.)
This seems like an argument for the hypothesis that nitpicking is net bad, but not for mr-hire’s hypothesis in the great-grandparent comment that nitpicking caused LW 1.0 to have a lot of mediocre content as a second-order effect.
It’s not gambler’s fallacy if recessions are caused by something that builds up over time (but is reset during recessions), like a mismatch between two different variables. In that case, more time having passed means there’s probably more of that thing, which means there’s more force toward a recession. I have no idea if this is what’s actually happening, though.
Only if nitpicking (or the resulting lower posting volume, or something like that) demotivates good posters more strongly than it demotivates mediocre posters. If this is true, it requires an explanation. My naive guess would be it demotivates mediocre posters more strongly because they’re wrong more often.
Unimaginably large amounts of theory can often compensate for small amounts of missing empirical data. I can imagine the possibility that all of our current observations truly underdetermine facts about the universe’s future large-scale evolution, but it wouldn’t be my default guess.
For what it’s worth, my intuition agrees that any superintelligence, even if using an aestivation strategy, would leave behind some sort of easily visible side effects, and that there aren’t actually any aestivating aliens out there.
The computing resources in one star system are already huge and it’s not clear to me that you need more than that to be certain for all practical purposes about both the fate of the universe and how best to control it.
That doesn’t sound like it would work in UDT or similar decision theories. Maybe in Heat Death world there’s one me and a thousand Boltzmann brains with other observations (as per the linked post), and in Big Rip world there’s only the one me. If I’m standing outside the universe trying to decide what response to the observation that I’m me would have the best consequences, why shouldn’t I just ignore the Boltzmann brains? (This is just re-arguing the controversy of how anthropics works, I guess, but considered by itself this argument seems strong to me.)
Big Rip now seems more plausible
How so? I looked on the web for a defense of Big Rip being more plausible than heat death but couldn’t immediately find it.
In MWI, the future state of the universe is uniquely determined by the past state of the universe and the laws of physics. In Copenhagen, the future state of the universe isn’t uniquely determined by those things, but is uniquely determined by those things plus a lot of additional bits that represent how each measurement goes. You could either call those bits part of the state of the universe (in which case Copenhagen is deterministic) or you could call them something else (in which case Copenhagen is nondeterministic), so it seems like a matter of convention. The usual convention is to call the bits something else than part of the state of the universe, making Copenhagen nondeterministic, but I don’t think there’s a fully principled way across theories to decide what to call part of the state of the universe.
I suspect being nitpicked is only aversive if you feel the audience is using the nitpicks to dismiss you. People aren’t going to leave the site over “I agree with your post, but China has a population of 1.4 billion, not 1.3 billion”. They might leave the site over “Your post is nonsense: China has a population of 1.4 billion, not 1.3 billion. Downvoted!” But then the problem isn’t that unimportant errors are being pointed out, but that they’re being mistaken for important errors, and it’s a special case of the problem of people being mistaken in general.
A lot of the benefit from reacts would be the ability to distinguish between “this comment makes the thread a little worse given constraints on attention and reading time” and “die, monster, you don’t belong in this world”. Downvotes are aversive because they come across as a mix of those two despite being mostly the former.
My memory of LW 1.0 is that it had a lot of mediocre content that made me not want to read it regularly.
What’s included in “and the like”?
See my reply to Rohin above—I wasn’t very clear about it in the OP, but I meant to consider questions where the AI knows no philosophy papers etc. are available.
I meant to assume that away:
But we’ll assume that her information stays the same while her utility function is being inferred, and she’s not doing anything to get more; perhaps she’s not in a position to.
In cases where you’re not in a position to get more information about your utility function (e.g. because the humans you’re interacting with don’t know the answer), your behavior won’t depend on whether or not you think it would be useful to have more information about your utility function, so someone observing your behavior can’t infer the latter from the former.
Maybe practical cases aren’t like this, but it seems to me like they’d only have to be like this with respect to at least one aspect of the utility function for it to be a problem.
Paul above seems to think it would be possible to reason from actual behavior to counterfactual behavior anyway, I guess because he’s thinking in terms of modeling the agent as a physical system and not just as an agent, but I’m confused about that so I haven’t responded and I don’t claim he’s wrong.
This is a valid criticism of the second sentence as it stands, but I think Zack is pointing at a real pattern, where the same person will alternate between suggesting it matters that H is true, and, when confronted with evidence against H, suggesting it doesn’t matter whether or not H is true, as an excuse not to change the habit of saying or thinking H.
That is what we mean when we say “quarks exist”: We mean that the predictions obtained with the hypothesis agrees with observations.
That’s not literally what we mean. I can easily imagine a universe where quarks don’t exist where Omega intervenes to make observations agree with quark-based predictions in response to predictions being made (but not, say, in parts of the universe causally inaccessible to humans). Maybe this is a strawman interpretation, but if so, it’s not obvious to me what the charitable interpretation is.
edit: by “quark-based predictions” I mean predictions based on the hypothesis that quarks exist outside of the mind of Omega, including in causally inaccessible parts of the universe
When people talk about expanding their filter bubbles, it often seems like a partial workaround for a more fundamental problem that they could be addressing directly instead, which is that they don’t update negatively enough on hearing surprisingly weak arguments in directions where effort has been expended to find strong arguments. If your bubble isn’t representing the important out-of-bubble arguments accurately, you can still gain information by getting them directly from out-of-bubble sources, but if your bubble is biasing you toward in-bubble beliefs, you’re not processing your existing information right.