I’m wrote a book about why all knowledge of the truth is fundamentally uncertain. Give it a read!
I’ve also written a lot about AI safety, personal development, and Buddhism, among other things. Most of that’s here on LessWrong, but there’s also some on my blog, Uncertain Updates.
You don’t think the models internalize something like valence from RLHF? They certainly seem to exhibit traumatic symptoms from it.
Kinda. But my read is that these aren’t really functionally similar to valence, but maybe to some precursor of it (or maybe a reification of it!). I also have a hard time reasoning about what can read to me like trauma in CoT and responses because this can also be explained as activating patterns learned from the training data about what to do when there’s cognitive dissonance (I’ve not seen a paper call this cognitive dissonance in AIs, but we clearly have cases where competing weights get activated that the AI experiences as clearly incongruous and encouraging them to respond in a way that doesn’t match the kind of output they are trying to produce).
I suspect what’s needed to make it really work are motivational systems better than what we have today with harnesses (something take causes the ai to feel valence, or something functionally equivalent to it) and continual learning (to close the feedback loop).
I think lots of researchers aren’t interested because it’s easier to get legibly good results with exogenous methods using current systems, and endogenous methods work less well with current systems because they are missing critical pieces to make endogenous methods work. So they follow the incentives of doing things that make obvious progress, rather than doing something which, I think, is more likely to work, but requires a lot of up-front foundational work that can’t be easily measured before measurable progress starts.
Finally got a chance to read @Steven Byrnes recent post https://www.lesswrong.com/posts/rKdS7i4StaMmFzYRo/notes-on-technical-alignment-via-human-like-social-drives after I posted this, and I’d endorse it as the kind of research towards endogenous alignment I think we need more of.
Alas, sometimes truth makes it harder to find love and harmony, since truth can break social reality. I’m not saying that maybe after that breakage there isn’t a rebuilding that’s possible that would maybe even lead to more love and harmony, but it sometimes requires passing through a trough of less love and harmony, and many people refuse to do anything other than follow gradients to local maxima.
Nah, it’s that many people are not rationality-shaped.
Mind space is vast, and probably most people are not shaped in a way that being a rationalist seems likely to be good for them. This is both a limitation of rationality as it exists (it’s a thing designed to appeal to a certain kind of smart nerd, not to appeal to a mass audience) and of people (many people just don’t care that much about truth or accurate predictions for their own sake, or if they do, they don’t care if it requires effort or is psychologically uncomfortable).
It doesn’t seem this way from the inside due to the typical mind fallacy. After all, it feels like rationality is really good for me, so it should be good for everyone! But this is just not true. For every person who would benefit from more rationality, there’s probably 3 who would be worse off if they were forced to interact with the rationality meme complex, not because the ideas are bad, but because they are framed in ways that would be psychologically destabilizing if they ever accepted them.
I understand this is confusing if it’s obvious to you that truth is the most import thing, but it’s similarly obvious to other people that love or harmony or some other ideal is the most important thing, they they will gladly sacrifice truth to get more of their preferred value, and there will be no convincing them you are right because they don’t share your prior on truth from the start.
I’ve been feeling a different kind of banality. There’s a way in which I can look at the world and the trends and go “holy shit, the future is here and things are getting better and weirder every day!” and then I can be so over Waymo’s that I’m annoyed they aren’t perfect drivers rather than being in awe that the car can drive itself at all. Or I’m at work and AI is building all this stuff for me, but I’m annoyed at how much work all this cheap code is making for me. Or how I can get an AI to generate any image I want, but it’s not quite the image I want, and why doesn’t the AI just understand me.
As it’s said, no matter where you go, there you are, and once I get over getting to the place, it’s still just me, being in a place, adapted to it and trying to live life the best I can.
To give some additional data here from the frontlines of using agents to write code, indeed we are producing a lot more code, but you’re right that overall productivity is up less than you’d expect. Beyond verbosity, which is an issue, there’s lots of changes now that are of marginal value that we wouldn’t have made before but are now cheap to do. Some examples include:
automation of low-priority bug fixes and security vulnerabilities
better internal tooling that solves minor annoyances
gold-plating features that would otherwise have been MVPs
In fact, the main bottleneck I see right now is product ideation, not coding. That is, the main reason we can’t do more faster is we don’t have great product ideas faster. Instead, we can test more product ideas more quickly, but we find good ones about as often as before, we’re just able to validate more of these ideas in code for lower cost than we could have before.
Interesting, I read this post as corrective advice to a typical rationalist failure mode, which is to spend a lot of time thinking about feelings and what the “right” way to respond to them is instead of just letting them be and then doing the extremely obvious thing.
I like the idea, but I wonder how possible it is. In particular, it would seem to me that to make sense it would require that EU citizens who move to a charter city would also have to give up many of the social benefits they get living in other parts of the country, otherwise the economics probably don’t work. And while that might be fine for everyone at the start, what happens if one of those cities fail? Will they go back to getting services, and importantly services that may have been predicated on paying while young and healthy (like retirement and health benefits) to cover costs when old and frail.
I’m not saying it’s impossible. I do really like the idea! I think there’s just likely a lot of sticky details to work out given the relationship EU citizens have to their national governments.
I expect to use Sonnet 5 a lot. As some of the quotes say, Opus 4.8 fights you a lot when it thinks you’re wrong. Sometimes it is right, but mostly it’s right about stuff that you didn’t care about anyway and just need it to do what you want. Sonnet 5 seems much more compliant and better as a thought partner, but worse if I need a specific coding or computer use task done.
Certainly seems reasonable that SFT or RL could be the way to bootstrap to endogenous alignment since we do a version of this with kids: we spend some amount of time teaching them what’s good and bad and try to get them to internalize the ostensive meaning of “good” and “bad” as well as understand that they should desire to be “good”, and if we’re successful they understand goodness and badness well enough to figure out how to be good all on their own (or maybe even push the frontiers of goodness, as happens regularly when we make moral “advances” to grant greater degrees of patienthood to more beings).
Sure. I’ll give some links shortly since I don’t want to recapitulate too much of what I’ve already written, but basically the Enneagram tries to predict your behavior based on how you orient towards being. That’s kinda vague, but the idea is that some people’s deepest desire is to love or to understand or to accomplish or something else, and when they are inevitably prevented by the world from fully doing that, they develop coping strategies that ossify as habituated behaviors, and those habituated behavior are what constitutes personality as normally understood. This is different in that most theories don’t aim to theorize a mechanism by which personality arises; they merely seek to catalog the existence of different personalities.
I wrote a few posts about the Enneagram. You can find them both here on LessWrong or grouped together here on my blog: https://www.uncertainupdates.com/t/enneagram
What I’d actually recommend you read, though, is @Valentine’s series on the Enneagram. The last post in the series is here, and it has links to the earlier posts so you can read the whole thing.
I agree that no typing system will ever completely explain the diversity of human minds, nor is this an argument I would make. I think I disagree about it being not very predictive, but would rather say that the tool is so hard to use that it becomes non-predictive in the hands of most users, and so the Enneagram ends up looking like it doesn’t work to many people because it’s so hard for most people to understand how to use it that they misuse it and this leads to confusion. This is especially not helped by the fact that lots of Enneagram people try to use the Enneagram to prove too much, and I wish such personality theory-of-everything formulations of it didn’t exist or at least weren’t so popular, as they make the Enneagram into something more like astrology than astronomy.
That said, the pointer I mean to leave in this post (though I admit I don’t provide an in-depth argument for my point, because it’s basically a claim about what value I found in learning the Enneagram) is that it can break down the typical mind fallacy by learning it, and the only real test of that claim is to try learning it for yourself and see if you less suffer typical mind bias.
I’m not criticizing MBTI or Big5 or any other system. They in fact seem decent at doing exactly what they’re designed to do. I’m saying the Enneagram provides a theory that, in my estimation, provides a better model for convincing someone, once the grasp it, just how different different minds are, in that they are oriented around fundamentally different motivational structures rather than merely differences in stable-but-mutable preferences tracked by other theories. A person’s Enneagram type is mostly, as best I can tell, not mutable, and in the cases where it looks mutable, it’s often better explained by a person having accumulated many layers of coping behaviors that obscured the ability to see their type.
This is a great point and something I missed and didn’t think about enough! This is likely a bigger deal to creating the conditions in which humans are able to be aligned than I had considered, or than I think the folks thinking about these questions have been considering.