Same skill, applied on multiple levels. The skill in “becoming rational”/”coordinating groups of humans”/”aligning AI” is all skill in alignment.
I see what these words might say, but don’t follow the link. Like, seems basically true that rationalism → human coordination works, but AI alignment is such a different thing, so alien to whatever concepts help humans self-align and coordinate.
Perhaps I just need more time to work through this concept. Right now I’m more focused on understanding my own mind, to make better decisions, because I’m finding a crapton of low-hanging fruit very quickly.
I’ve rolled this around in my head for a bit, and it seems to me like, for rationality, “control” of lower processes is better done by something like “improved training data” than operant force.
This is an vignette from my life, not a quote from anyone:
I’ve noticed since I was about 16 that I get sad when seeing attractive women. This always sucked! I tried introspecting, but I was looking at the feeling of sadness (and often trying misguidedly to control that feeling).
Two days ago, I started looking at what the other parts of my mind were doing when I see women; what am I pulling towards, where’s the tension? Oh, partnered romance feels unreachable? Where’s that coming from?
It mostly seems to me that this (unreachable romance) is the wrong conceptualization. Like, my non-deliberate processes are using the wrong concepts. My intuitive ontology is factually wrong, in the sense I care about; and this wrongness tied itself into a self-perpetuating loop (what I’m calling a cognitive attractor).
And so most of the art of rationality, in fact possibly exactly all of it, is to intuitively correct these errors:
I can in fact date, I’ve been accidentally choosing not to
It doesn’t actually hurt me to admit I’m wrong, in most cases
Eyedrops aren’t scary :D
because imposing top-down “control” misaligns:
The optimizer which calls itself an Elliot, wants human flourishing, wants to hold someone
Anyway, your post on BCI facilitated AI alignment looks to me like a step in the same direction. A step towards noticing that AI alignment is downstream of human alignment (in this case, because aligned and augmented humans are more competent which is instrumentally useful), and that the solutions which actually work have more competent humans more tightly integrated in the alignment process for longer
Sounds plausible
rather than keeping a stance of “I’m outside the system, aligning THAT THING is what I’m trying to do, dammit”.
I very much have come to think of myself as a system, nothing special other than this weird consciousness phlogiston, though I haven’t stopped using self/other borders.
Does this fit?
Maybe? I find myself confused about your explanation of AI alignment; but after reading Valentine’s post about memeplexes, I’m thinking you’re talking about being in entirely the wrong frame, where “align the AI” straightforwardly might not be a thing. And “stop doom” might also not be a thing.
(Thanks for linking that, by the way. I preliminarily do expect, if BCIs or similar enhancements work, that moderately superintelligent humans will birth more powerful Friendly hypercreatures. If you have a list of older, similarly gearsy posts, I’d definitely like to read them.)
I see what these words might say, but don’t follow the link. Like, seems basically true that rationalism → human coordination works, but AI alignment is such a different thing, so alien to whatever concepts help humans self-align and coordinate.
Perhaps I just need more time to work through this concept. Right now I’m more focused on understanding my own mind, to make better decisions, because I’m finding a crapton of low-hanging fruit very quickly.
I’ve rolled this around in my head for a bit, and it seems to me like, for rationality, “control” of lower processes is better done by something like “improved training data” than operant force.
This is an vignette from my life, not a quote from anyone:
because imposing top-down “control” misaligns:
The optimizer which calls itself an Elliot, wants human flourishing, wants to hold someone
vs the constellation of smaller optimizers Elliot is an intelligible supervenience of
Sounds plausible
I very much have come to think of myself as a system, nothing special other than this weird consciousness phlogiston, though I haven’t stopped using self/other borders.
Maybe? I find myself confused about your explanation of AI alignment; but after reading Valentine’s post about memeplexes, I’m thinking you’re talking about being in entirely the wrong frame, where “align the AI” straightforwardly might not be a thing. And “stop doom” might also not be a thing.
(Thanks for linking that, by the way. I preliminarily do expect, if BCIs or similar enhancements work, that moderately superintelligent humans will birth more powerful Friendly hypercreatures. If you have a list of older, similarly gearsy posts, I’d definitely like to read them.)