The first time AI reaches STEM+ capabilities (if that ever happpens), it will disempower humanity within three months
So this is asking for P(fasttakeoff and unaligned | STEM+) ? It feels weird that it’s asking for both. Unless you count aligned-AI-takeover as “disempowering” humanity. Asking for either P(fasttakeoff | STEM+) or P(fasttakeoff | unaligned and STEM+) would make more sense, I think.
Do you count aligned-AI-takeover (where an aligned AI takes over everything and creates an at-least-okay utopia) as “disempowering humanity”?
“reasonable and informed” is doing a lot of work here — is that left to be left to the reader, or should there be some notion of what rough amount of people you expect that to be? I think that, given the definitions I filled my chart with, I would say that there are <1000 people on earth right now who fit this description (possibly <100).
I really like this! (here’s mine)
A few questions:
So this is asking for P(fasttakeoff and unaligned | STEM+) ? It feels weird that it’s asking for both. Unless you count aligned-AI-takeover as “disempowering” humanity. Asking for either P(fasttakeoff | STEM+) or P(fasttakeoff | unaligned and STEM+) would make more sense, I think.
Do you count aligned-AI-takeover (where an aligned AI takes over everything and creates an at-least-okay utopia) as “disempowering humanity”?
“reasonable and informed” is doing a lot of work here — is that left to be left to the reader, or should there be some notion of what rough amount of people you expect that to be? I think that, given the definitions I filled my chart with, I would say that there are <1000 people on earth right now who fit this description (possibly <100).