Yeah, Drexler was obviously-to-a-polymath on a separate track from Feynman, and Feynman was the distant prestigious person who said a thing helpful to Drexler but wasn’t remotely being Drexler about it.
Eliezer Yudkowsky
Karma: 170,259
Martingales or personal probability assignments?
You Heard of “Bayesians Don’t Predictably Update” Because of Robin Hanson
I am a straightforward person. If I tell someone “not sure what you mean”, it means: I wasn’t sure what he meant.
I looked at what Epstein had said and went ????? and guessed it was a random billionaire thing about, “As a random billionaire, I prefer not to have it spread around where I am interested in donating and will probably go through a DAF; what are you doing with my name / who are you telling my name to?”
I lacked context.
Well, that’s one way for an older generation to die out, and a new generation to grow up familiar with the new ideas.
How would you characterize the basic, underlying difficulties you think face your theoretical program?
If Anyone Builds It, Everyone Dies: One Year Closer
If intelligent life is spectacularly rare, we should look around ourselves and see an older universe, rather than the sort of young universe that no other intelligent life has had time to reach and colonize.
You seem confused; I can’t make out what claims you think you’re addressing or contradicting. I did not say the Swarm thought they were being graded by an LLM. Did anyone else think I said that?
Thank you for recounting item 4. I think this is an important point to propagate.
The Talker Does Not Control The Doer (in Current AIs)
I have tried bolding the contestants; I wonder if that helped by way of contrast?
Then we should look around ourselves and see an older universe.
Also this falls under the category of “distinguish the possible from the probable”. No species anywhere cares about leaving one little probe behind to stop Holocausts?
Can a superintelligence do THAT?
Thanks for this high-priority correction, I’ll edit.
Now is when you’ve got leverage. In 3-24 months threats to leave will have zero leverage on them because they no longer need additional employees.
I surely do credit with wisdom those EAs who went off and quietly bought bednets and saved a few kids, rather than posting to the Internet their arguments for why MIRI couldn’t possibly be a better bet than bednets. The difference between actual humility and the posturing that is Modesty.
The ones endlessly going “But what if there’s a mistake in your argument?” are the ones I’m rolling my eyes at, here. Well, golly fuck, what if there’s a mistake in your fucking argument?
Now the quiet modest ones who bought bednets and didn’t fuck around in difficult-to-analyze grownup business are going to have their work undone when the children in Africa die anyway, because other people borrowed their name and used that reputational credit to, among other things, shit all over the attempts to have Earth do something other than die.
A hope for debate might be that, if a person has a good-enough hidden stat to construct an adequate alignment plan in the traditional way (reading, thinking, corresponding with other humans, etc.) given enough time
They don’t. They were not converging toward correct answers; their updates from debate were not moving in a correct direction. So far as I can tell, EA given unlimited time to think, but no further evidence, would’ve not figured out that AGI wasn’t scheduled for 2050.
Looks like you got your key insight down to one sentence, rather than needing to start out with a long description of a complicated mechanism that would choose eleven mice for some unknown reason that you could only explain later!
It’s talent-constrained, but not in a form that the current institutions can recognize or have short-term incentives to select for, so none of those $402K jobs are going to be available to the actual talent required.