Alternatively, “lawful universe” has lower Kolmogorov complexity than “lawful universe plus simulator intervention” and thereore gets exponentially more measure under the universal prior?? (See also “Infinite universes and Corbinian otaku” and “The Finale of the Ultimate Meta Mega Crossover”.)
Zack_M_Davis(Zack M. Davis)
Karma: 6,153
Why appeal to philosophical sophistication rather than lack of motivation? Humans given the power to make ancestor-simulations would create lots of interventionist sims (as is demonstrated by the populatity games like The Sims), but if the vast hypermajority of ancestor-simulations are run by unaligned AIs doing their analogue of history research, that could “drown out” the tiny minority of interventionist simulations.
If X are making claims that everyone knows are false, then there’s no element of deception
“Everyone knows” is an interesting phrase. If literally everyone knew, what would be the function of making the claim? How do you end up with a system that wouldn’t work without false assertions, and yet allegedly “everyone” knows that the assertions are false? It seems more likely that the reason the system wouldn’t work without false assertions, is because someone is actually fooled. If the people who do know are motivated to prevent it from becoming common knowledge, “It’s not deceptive because everyone knows” would be a tempting rationalization for maintaining the status quo.
if you donate masks today I expect them to be used much more quickly than if you wait and donate them when things are worse.
But this (strategically timing your donation because you don’t expect the recipient to use the gift intelligently if you just straightforwardly gave when you noticed the opportunity) is kind of a horrifying situation to be in, right? If you can see the logic of the argument, why can’t hospital administrators see it, too?—at least once it’s been pointed out?
This reminds me of that cute information-theory puzzle about finding the ball with the different weight! I’m pretty dumb and bad at math, but I think the way this works is that since each test is a yes-or-no question, we can reduce our uncertainty by at most one bit with each test, and, as in Twenty Questions, we want to choose the question such that we get that bit.
A start on the simplest variant of the problem, where we’re assuming the tests are perfect and just trying to minimize the number of tests: the probability of at least one person in the pool having the ’roni in pool size S is going to be , the complement of the probability of them all not having it. We want to choose S such that this quantity is close to 0.5, so . (For example, if P = 0.05, then S ≈ 13.51.)
My next thought is that when we do get a positive test with this group size, we should keep halving that group to find out which person is positive—but that would only work with certainty if there were exactly one positive (if the “hit” is in one half of the group, you know it’s not in the other), which isn’t necessarily the case (we could have gotten “lucky” and got more than one hit in our group that was chosen to have a fifty-fifty shot of having at least one) …
Partial credit???
We should trust the professionals who predict these sorts of things.
What? Why? How do you decide which professionals to trust? (Nick Bostom is just some guy with a PhD; there are lots of those, and most of them aren’t predicting a robot apocalypse. Eliezer Yudkowsky never graduated from high school!)
The reason I’m concerned about existential risk from artificial intelligence, is because the arguments actually make sense. (Human intelligence has had a big impact on the planet, check; there’s no particular reason to expect humans to be the most possible powerful intelligence, check; there’s no particular reason to expect an arbitrary intelligence to have humane values, check; humans are made out of atoms than can be used for other things, check and mate.)
If you think your audience just isn’t smart enough to evaluate arguments, then, gee, I don’t know, maybe using a moment of particular receptiveness to plant a seed to get them to open their wallets to the right professionals later is the best you can do? That’s a scary possibility; I would feel much safer about a fate of a world that knew how to systematically teach methods of thinking that get the right answer, rather than having to gamble on the people who know how to think about objective risks also being able to win a marketing war.
Why Telling People They Don’t Need Masks Backfired
It’s not an especially accurate depiction of royalty
You mean Tangled: The Series lied to me?!
The un-referenceable may, at best, be inferred (although, of course, this statement is absurd in refererring to the un-referenceable).
Would you also say that a lot of mathematics is absurd in this sense? For example, almost all real numbers are un-nameable (because there uncountably many real numbers, but only countably many names you could give a number).
Some authors use ostensive to mean the same thing as “extensional.”
As is demonstrated by the Hashlife algorithm, that exploits the redundancies for a massive speedup. That’s not possible for things like SHA-256 (by design)!
The quoted sentence claims that karma systems are a check against the unilateralist’s curse specifically, not infohazards in general, as is made explicit in the final sentence of that paragraph (“Conversely, while a net-upvoted post might still be infohazardous [...]”).
I’ve been envisioning “unilateralist’s curse” as referring to situations where the average error in individual agents’ estimates of the value of the initiative (what I called E in the post, but Bostrom et al. calls the error d and says it’s from a cdf F(d)) is zero, and the harm comes from the fact that the variance in error terms makes someone unilaterally act/veto when they shouldn’t, in a way that could be corrected by “listening to their peers.” If the community as a whole is systematically biased about the value of the initiative, that seems like a different, and harder, problem.
I think you should read Bostrom’s actual paper
Thanks for the suggestion! I just re-skimmed the Bostrom et al. paper (it’s been a while) and wrote up my thoughts in a top-level post.
that the reference class isn’t
Here we face the tragedy of “reference class tennis”. When you don’t know how much to trust your own reasoning vs. someone else’s, you might hope to defer the historical record for some suitable reference class of analogous disputes. But if you and your interlocutor disagree on which reference class is appropriate, then you just have the same kind of problem again.
The Heckler’s Veto Is Also Subject to the Unilateralist’s Curse
the Unilateralists’s curse
The underlying statistical phenomenon is just regression to the mean: if people aren’t perfect about determining how good something is, then the one who does the thing is likely to have overestimated how good it is.
I agree that people should take this kind of statistical reasoning into account when deciding whether to do things, but it’s not at all clear to me that the “Unilateralist’s Curse” catchphrase is a good summary of the policy you would get if you applied this reasoning evenhandedly: if people aren’t perfect about determining how bad something is, then the one who vetoes the thing is likely to have overestimated how bad it is.
In order for the “Unilateralist’s Curse” effect to be more important than the “Unilateralist’s Blessing” effect, I think you need additional modeling assumptions to the effect that the payoff function is such that more variance is bad. I don’t think this holds for the reference class of “blog posts criticizing institutions”? In a world with more variance in blog posts criticizing institutions, we get more good criticisms and more bad criticisms, which sounds like a good deal to me!
I wrote a big critique outlining why I think it’s bad, but I couldn’t keep it civil and don’t want to spend another hour editing it to be
If you post it anyway (maybe a top-level post for visibility?), I’ll strong-upvote it. I vehemently disagree with you, but even more vehemently than that, I disagree with allowing this class of expense to conceal potentially-useful information, like big critiques. (As it is written of the fifth virtue, “Those who wish to fail must first prevent their friends from helping them.”)
I’m really not trying to make anyone feel bad
Shouldn’t you? If the OP is actually harmful, maybe the authors should feel bad for causing harm! Then the memory of that feeling might stop them from causing analogous harms in analogous future situations. That’s what feelings are for, evolutionarily speaking.
Personally, I disapprove of this entire class of appeals-to-consequences (simpler to just say clearly what you have to say, without trying to optimize how other people will feel about it), but if you find “This post makes the community harder to defend, which is bad” compelling, I don’t see why you wouldn’t also accept “Making the authors feel bad would make the community easier to defend (in expectation), which is good”.
A “no canceling anyone” promise isn’t very valuable if most of the threat comes from third parties—if you’re afraid to talk to me not because you’re afraid of attacks from me, but because you’re afraid that the intelligent social web will attack you for guilt-by-association with me. A confidentiality promise is more valuable—but it’s also a lot more expensive. (I am now extremely reluctant to offer confidentiality promises, because even though my associates can confidently expect me to not try to use information to hurt them, I need the ability to say what I’m actually thinking when it’s relevant and I don’t know how to predict relevance in advance; there are just too many unpredictable situations where my future selves would have to choose between breaking a promise and lying by omission. This might be easier for people who construe lying by omission more narrowly than I do.)
I haven’t been able to find any discussion on LW about this.
I discuss this in “Heads I Win, Tails?—Never Heard of Her” (“Reality itself isn’t on anyone’s side, but any particular fact, argument, sign, or portent might just so happen to be more easily construed as “supporting” the Blues or the Greens [...]”).
Richard Dawkins seemed surprised
I suspect Dawkins was motivatedly playing dumb, or “living in the should-universe”. Indignation (e.g., at people motivatedly refusing to follow a simple logical argument because of their political incentives) often manifests itself as expression of incomprehension, but is distinguishable from literal incomprehension (e.g., by asking Dawkins to bet beforehand on what he thinks is going to happen after he Tweets that).
Ordinary computer programmers don’t do this. (As it is written, “move fast and break things.”) What will spur AI developers to greater caution?