I wonder if he’d just memorised the first couple dozen digits of something like Chaitin’s constant or e or pi like you or whatever and just started somewhere in the middle of his memorised substri
Yeah that’s what I suggested people try if they want a near-perfect external source of randomness.
Yeah I suspect a distilled/pruned neural-net could do really well against the existing strategies, I dunno what the latest state of the field is. I also don’t have a very good sense of recent progress in computer/ML optimizations so maybe there are other tricks I’m discounting.
When I remember to, I try to keep my fist in a “neutral” state (thumb not touching fist so it’s easy to either close or open) until after my hand starts moving, but I do wonder how common hand-shape prediction is, and if so, how often high-level players try to “bait.”
I don’t take the game very seriously outside of the bot exercises and I’ve never played in irl tournaments.
Thanks for being nice about the whole thing! Would totally understand if you’re really angry given the context etc.
Interesting, are there examples of people feeling positive-tribal-emotions to “go insane” in the abstract?
I suspect in practice it looks more like social pressure to be sleep-deprived, social pressure to repeat what the Glorious Leader says without question, pressure to ignore widely held taboos, pressure to sacrifice the self, etc.
Yeah I thought about including randomness exploits for bots too (instead of just noting it for humans) but decided it’d be distracting.
Hopefully if I ever run that contest I’ll be able to catch problems like randomness exploits and forced crashes!
Thanks for the feedback!
Hmm my general take on this is that war is bad! It is frequently irrational, and always destructive I’d like there to be less war. Conditional upon there being war however, I think the sadness of weapons causing people to die is sort of “priced in”[1], whereas the death of the bats was idiosyncratic and particular to stealth technology specifically.
I do get what you’re saying, and I apologize if my post was insensitive. For what it’s worth, my impression is that other things I’ve seen covering weapons are substantially more cold-minded, and my full review of Skunk Works will investigate the ethical issues of working in the US military-industrial complex more thoroughly.
Though I suppose you could make the argument that stealth is more offense dominant than defense dominant, unlike say trench-lines.
Sure but at some point your probability of murder goes below that of suicide, no? It seems like your argument would imply that you can never think suicide is more likely than murder based on investigating all the plausible ways it could be murder, unless you have an actually exhaustive search or an airtight logical argument.
Similarly, re: the Fatima sun case, I think what Ethan did was that he investigated all the ways he thought it could be natural, until his probability of natural explanation was lower than his probability that it was a miracle. At face value, this seems like a perfectly reasonable method of investigation to me! To the extent there was a reasoning mistake, I think of it as very different from how you’re conceiving it. I think he had a) too high a prior for supernatural explanations, b) did not sufficiently adjust for the selection bias in choosing the most plausible miracle (Where the correct update is that if the most plausible miracle is <X persuasive, you should update downwards on God), and c) was overly credulous in one direction than the other.
In the Rethink and superforecaster cases, I don’t think what you said is a particularly plausible account of what they did to begin with.
David Graeber is not saying that all middlemen are bad. His thesis is that middleman jobs where the person who has the job believes the job is useless are bad.
I think alienation is real but exaggerated, and we also shouldn’t take people’s alienation as substantial evidence their work is actually meaningless.
The argument here feels somewhat slippery, and I think I’m overall not convinced by this.
If you’re trying to investigate a crime, it seems like the best process is to go through all the most likely leads until your marginal opportunity cost of further investigation exceeds the marginal benefit. Do you a) disagree that this is an appropriate way to investigate a crime? or do you b) disagree that this process is analagous to what you call “Exhaustive Free Association?”
Now of course a genuinely exhaustive search would be better. But you can’t usually realistically search every person as a potential suspect, nor rule people out in broad categories (Here’s the case for a male criminal, here’s the case for a female criminal). So in practice I don’t think you can do better than a non-exhaustive search.
Additionally, the specific examples you choose both seem not very strong, and tilted by your object-level commitments.
But honestly? The best defence is to make your argument better. If you’re relying on something which looks like an exhaustive free association, your first worry should be that it actually is! Invert the argument to a set of assumptions. Go up a level to a general principle. Go down a level to specific cases.
This paragraph in particular sounds very AI-assisted to me.
Re
I’m not convinced though that ASI will bother to kill us or, if it does, very immediately.
I don’t think we’re certainly doomed (and have shallower models than Eliezer and some others here), but for me the strongest arguments for why things might go very badly:
An agent that wants other things might find their goals better achieved by acquiring power first. “If you don’t know what you want, first acquire power.” Instrumental convergence is a related concept.
There is and will continue to be strong training/selection effects for agency and not just unmoored intelligence for AI in the upcoming years. Ability to take autonomous actions is both economically and militarily useful.
In a multipolar/multiagent setup with numerous powerful AIs flying around, the more ruthless ones are more likely to win and accumulate more power. So it doesn’t matter if some fraction of AIs wirehead, become Buddhist, are bad at long-term planning, have very parochial interests etc, as long as some powerful AIs want to eliminate or subjugate humanity for their purposes, and the remaining AIs/rest of humanity don’t coordinate to stop them in time.
This arguments are related to each other, and not independent. But note also that they don’t have to all be true for very bad things to happen. For example, even if (2) is mostly false and labs mostly make limited, non-agentic, AIs, (3) can still apply and a small number of agent ASIs roll over the limited AIs and humanity.
And of course this is not an exhaustive list of possible reasons for AI takeover.
Some nuances I decided not to include in the main post:
1. The first stealth airplanes were mostly dominated by shape by not entirely. They also used radar absorbing materials. Maybe 3 OOMs shape, 1 OOM absorption was an estimate given by an expert. So shape gets most of the Shapley but not entirely.
2. Flat surfaces were the main design principle of first-gen stealth airplanes but modern software have allowed people to figure out curved surfaces that still doesn’t reflect radio echoes to the sender-receiver. Still, if you look at pictures of modern stealth airplanes they still look much more angular and flat than old airplanes, or passenger aircraft.
3. Modern stealth airplanes and radar detection have to adapt in a sort of Red-Queen race situation so the simple bouncing technique doesn’t fully work anymore. I didn’t closely investigate why.
4. I simplified stealth as “evading radar” but of course there are other detection methods (sight, sonar, thermal detection). Eg reducing your jet planes’ heat signatures is important to modern stealth as well. But I didn’t think it’s as relevant to the main argument.
5. My argument/analysis here works for any sender-receiver style. Eg it also works for echolocation and bats (interestingly, and sadly in my view, bats often die by killing themselves on stealth airplanes). Radar has structural advantages over sonar (sound waves) and light. This is why militaries use them. I didn’t bother clarifying this as it is not relevant to the core of stealth technology.
Traditionally spy aircrafts fly at night, exactly for this reason. I suppose you could also paint them sky-blue for camouflage but I think empirically “flying at night” and being grey-black is the historical strategy that they’ve chosen.
The third way I stay sane is a fiat decision to stay sane.
My mental landscape contains that option; I take it.
This is the point I am even less expecting to be helpful, or to correspond to any actionable sort of plan for most readers.
Some years ago, I had a friend who told me she was still anorexic even though the reason she originally acquired anorexia no longer applies[1].
I responded “Have you considered not being anorexic?” She thought about it and replied something like “No, actually.”
Two weeks later she thanked me for helping to cure her anorexia.
This is the type of advice that I expect to be profoundly unhelpful to >95% of people in that position (and indeed is rightfully lampooned approximately everywhere). Yet it was the exact thing this specific person needed to hear, and hopefully “you can just decide to stay sane” is the exact thing some small fraction of people reading your post needed to hear as well.
(censoring the exact reason)
I have a number of notes on questions I’m interested in. Would love to see some feedback and further thoughts from people!
I mostly agree with this post. I wrote some notes, trying to understand and extend this idea. Apologies for the length which ended up being longer than your post while having less important content, I was a bit rushed and therefore spent less time making my post succinct than the topic deserved.
What model?
I think this discussion is very hard to have because people have substantial variation in how good their base intuitions, including social intuitions, are and also substantial variation in how good their rational minds (S2′s) are.
Most good forecasters start off with pretty good base intuitions for probabilities, and forecasting/calibration practice helps refine their intuitions to be even better. Some good forecasters start with terrible probabilistic intuitions and the formal exercises help improve their intuitions. (I suppose it’s also theoretically possible for some good forecasters to start with terrible probabilistic intuitions, continue to have terrible probabilistic intuitions, but forecast well because their formal practice and analysis allows them to override their intuitions. I’ve just never heard about this in practice).
Many mental health conditions come from creating a wrong intuition/prior about the world that rational-adjacent therapy methods like CBT teach you to override. Depression being the most famous/common. When I’m depressed, it’s natural to fixate on specific high-noise signals and automatically assume bad interpretations of the evidence (eg someone didn’t respond to a text because I’ve upset them, or they don’t like me for other reasons). Presumably there’s the inverse problem as well (eg manics have a very positive “rose-colored goggles” prior for everything), but I’ve never heard of this as a serious problem in practice.
In the realm of social intuitions, I think there’s large variation in how good different social intuitions are. Autism in particular is a disorder which has a core manifestation of being bad at parsing social intuitions. I imagine most young autistic people would benefit from learning higher-order rationality and principles about social questions counteracting/overriding their own poor social intuitions (“vibes”), and to a lesser extent training/calibrating their poor social intuitions to become better. On the flip side, neurotypical people that are prone to ideological capture should potentially learn to trust their hearts more, and be less inclined to trust higher order ideologies and “rationality” (which often manifests as rationalizations), in favor of their base intuitions.
But even all of these comments are overly high-order glosses. Autism does not make you immune to rationalization, nor does ideological capture suddenly make you incapable of rationality. Reverse all advice you hear, etc.
oops! For some reason I brain-farted and thought it was a different constant I wasn’t familiar with.