Theoretical AI alignment (and relevant upskilling) in my free time.
Nicholas Kross
Karma: 2,053
From the MIRI announcement:
Our big ask for you is: If you have any way to help this book do shockingly, absurdly well— in ways that prompt a serious and sober response from the world — then now is the time.
sober response from the world
sober response
Uh… this is debatably a lot to ask of the world right now.
Something else I just realized: Georgism is a leftish idea that recognizes some (but not all) leftish ideas I’ve discussed or referenced above, and its modern form is currently rationalist-adjacent. Progress!
Ah, sorry yeah I think it was a mistake on my part to mostly make the post a verbatim Discord reply. Lots of high-context stuff that I didn’t explain well.
This specific part is (in my usage/interpretation; if you click the link, the initial context was an Emmett Shear tweet) basically a shorthand for one or more “basic” leftist views, along the lines of these similar-but-somewhat-distinct claims:
Capitalism more-reliably rewards power-maximizers than social-utility-maximizers.
Under capitalism and similar incentive-structures, we’d expect conflict theory to predict entities’ wealth better than mistake theory.
General outcomes, under capitalism and similar incentive-structures, are downstream of “brute power” (from guns to monopolies) far more than the things we’d “want” to reward (innovation, good service, helping people, etc).
In hindsight, I over-updated on my previous success with a poorly-written angry short post with a clickbait title and lots of inline links criticizing the rationality community. Oops.
I said “one of the best movies about”, not “one of the best movies showing you how to”.
Rationalist Movie Reviews
The punchline is “alignment could productively use more funding”. Many of us already know that, but I felt like putting a mildly-opinionated spin on what kind of things, at the margin, may help top researchers. (Also I spent several minutes editing/hedging the joke)
Virgin 2030s [sic] MIRI fellow:
- is cared for so they can focus on research
- has staff to do their laundry
- soyboys who don’t know *real* struggle
- 3 LDT-level alignment breakthroughs per weekCHAD 2010s Yudkowsky:
- founded a whole movement to support himself
- “IN A CAVE, WITH A BOX OF SCRAPS”
- walked uphill both ways to Lightcone offices.
- alpha who knows *real* struggle
- 1 LDT-level alignment breakthrough per decade
Kinda, my current mainline-doom-case is “some AI gets controlled --> powerful people use it to prop themselves up --> world gets worse until AI gets uncontrollably bad --> doom”. I would call it a different yet also-important doom case of “perpetual low-grade-AI dictatorship where the AI is controlled by humans in a surveillance state”.
EDIT: Due to the incoming administration’s ties to tech investors, I no longer think an AI crash is so likely. Several signs IMHO point to “they’re gonna go all-in on racing for AI, regardless of how ‘needed’ it actually is”.
For more details on (the business side of) a potential AI crash, see recent articles by the blog Where’s Your Ed At, which wrote the sorta-well-known post “The Man Who Killed Google Search”.
For his AI-crash posts, start here and here and click on links to his other posts. Sadly, the author falls into the trap of “LLMs will never get to reasoning because they don’t, like, know stuff, man”, but luckily his core competencies (the business side, analyzing reporting) show why an AI crash could still very much happen.
Further context on the Scott Adams thing lol: He claims to have taken hypnosis lessons decades ago and has referred to using it multiple times. His, uh, personality also seems to me like it’d be more susceptible to hypnosis than average (and even he’d probably admit this in a roundabout way).
I think deeply understanding top tier capabilities researchers’ views on how to achieve AGI is actually extremely valuable for thinking about alignment. Even if you disagree on object level views, understanding how very smart people come to their conclusions is very valuable.
I think the first sentence is true (especially for alignment strategy), but the second sentence seems sort of… broad-life-advice-ish, instead of a specific tip? It’s a pretty indirect help to most kinds of alignment.
Otherwise, this comment’s points really do seem like empirical things that people could put odds or ratios on. Wondering if a more-specific version of those “AI Views Snapshots” would be warranted, for these sorts of “research meta-knowledge” cruxes. Heck, it might be good to have lots of AI Views Snapshot DLC Mini-Charts, from for-specific-research-agendas(?) to internal-to-organizations(?!?!?!?).
I can’t make this one, but I’d love to be at future LessOnline events when I’m less time/budget-constrained! :)
First link is broken.
“But my ideas are likely to fail! Can I share failed ideas?”: If you share a failed idea, that saves the other person time/effort they would’ve spent chasing that idea. This, of course, speeds up that person’s progress, so don’t even share failed ideas/experiments about AI, in the status quo.
“So where do I privately share such research?” — good question! There is currently no infrastructure for this. I suggest keeping your ideas/insights/research to yourself. If you think that’s difficult for you to do, then I suggest not thinking about AI, and doing something else with your time, like getting into factorio 2 or something.
“But I’m impatient about the infrastructure coming to exist!”: Apply for a possibly-relevant grant and build it! Or build it in your spare time. Or be ready to help out if/when someone develops this infrastructure.
“But I have AI insights and I want to convert them into money/career-capital/personal-gain/status!”: With that kind of brainpower/creativity, you can get any/all of those things pretty efficiently without publishing AI research, working at a lab, advancing a given SOTA, or doing basically (or literally) anything that differentially speeds up AI capabilities. This, of course, means “work on the object-level problem, without routing that work through AI capabilities”, which is often as straightforward “do it yourself”.
“But I’m wasting my time if I don’t get involved in something related to AGI!”: “I want to try LSD, but it’s only available in another country. I could spend my time traveling to that country, or looking for mushrooms, or even just staying sober. Therefore, I’m wasting my time unless I immediately inject 999999 fentanyl.”
How scarce are tickets/”seats”?
I will carefully hedge my investment in this company by giving it $325823e7589245728439572380945237894273489, in exchange for a board seat so I can keep an eye on it.
https://www.lesswrong.com/posts/tpZciMYCXN49FYWnS/nicholaskross-s-shortform?commentId=f4PxFp8LkKKxdCXxh