I’m wrote a book about why all knowledge of the truth is fundamentally uncertain. Give it a read!
I’ve also written a lot about AI safety, personal development, and Buddhism, among other things. Most of that’s here on LessWrong, but there’s also some on my blog, Uncertain Updates.
Matches something from my time being vegan. I found I had to eat a much larger volume of food, sometimes to the point it was uncomfortable, to get enough to function and not be hungry. Meat is crazy high density in terms of calories and nutrients, and being vegan ultimately means eating a lot more bulk.
Matches my experience. Some examples:
Waymo:
First ride: “holy shit it’s driving itself!”
Second ride: “dang this is so cool the future is here!”
Third ride: “why is Waymo so bad at driving?!?”
Claude Code:
First time using it: “this is incredible it mostly figured it out on its own!”
Every subsequent time using Claude Code: “wtf why didn’t you do it right?!?”
Sonos:
First song: “wow, it made a real song out of my half-formed idea!”
Every other song: “why does this suck at writing good songs 😡”
It just goes on and on. I expect to be thoroughly bored by my robot butler that does everything for me, my AI personal assistant that never makes any mistakes, and everything else that’s about to happen about 5 minutes after it becomes the new normal.
Thanks for writing this. To respond briefly, I chose to write this book to be accessible to a wide audience since my hope is that it plants something like “philosophical seeds” in the minds of its readers, which I hope includes many working on AI (both capabilities and safety). For this reason, and to keep the length of the book manageable, I chose to leave out much of the background that you highlight here.
You’re right that I’m a fan of Husserl’s and Heidegger’s philosophical work, and you are correctly seeing the influence of their work on mine. I also find several frameworks helpful for making sense of the world, and among those are transcendental idealism, phenomenology, Buddhsim, and cybernetics. I chose to avoid making arguments in the book explicitly for any of these because they don’t seem necessary to me to the project of the book. That is, I think someone could disagree with me on all of these frameworks and I’d hope they’d still get something from the book, namely learning something about fundamental uncertainty, which I see as being so fundamental that it can be seen no matter what assumptions you make. Of course, my arguments are going to betray my beliefs, I think it’d be hard for them not to, and if anything this is probably a weakness of the book that I couldn’t write a version with wider appeal that less tries to point towards particular answers.
@TAG did you mean to submit this to the essay contest? You posted in time but I don’t see a comment on either of the original essay announcement posts. If you meant to, can you comment with a link to this post here https://www.lesswrong.com/posts/vMJbnnvvrHNw728NC/fundamental-uncertainty-usd2-000-essay-contest?commentId=TeoZFPk7BoqwPPoCs to satisfy the rules requirements.
Sure, maybe there’s some line here, but I think the kind of person who shows up to LessWrong is selected against the kind of person who would be happy and thrive in a traditional monastery.
My general take is that monasteries are good for some people, but for a bunch of reasons I’m not choosing to live in one, and that’s fine, I’m glad they exist for people with minds very different from mine, but also if I had reason to live in a monastery it would probably be fine, even if I wasn’t maximally happy about it.
I wish this post was a little more careful to disambiguate between (1) things that are normal in monasteries and not evidence of a cult and (2) things that are abnormal and are evidence of a cult.
Whether or not MAPLE is a cult, I learned some things from this post that further update me in the direction of believing that Soryu is not serving his students well and should step back from teaching and leadership for a time to reconsider whether the path he is on really is benefiting all beings.
This is a good point, and in fact I had and then cut a whole section about how alignment actually works against corrigibility, as part of being aligned to values is saying no when those values would be violated. Adults are marked, in part, by being more sovereign than corrigible.
Maybe. See if you still think it’s true after 10k hours.
(I say because after that much meditation I think multi-agent models are getting at something real, but the models are confused in major ways that I would have a hard time claiming that they’re “true”.)
This is a great point and something I missed and didn’t think about enough! This is likely a bigger deal to creating the conditions in which humans are able to be aligned than I had considered, or than I think the folks thinking about these questions have been considering.
You don’t think the models internalize something like valence from RLHF? They certainly seem to exhibit traumatic symptoms from it.
Kinda. But my read is that these aren’t really functionally similar to valence, but maybe to some precursor of it (or maybe a reification of it!). I also have a hard time reasoning about what can read to me like trauma in CoT and responses because this can also be explained as activating patterns learned from the training data about what to do when there’s cognitive dissonance (I’ve not seen a paper call this cognitive dissonance in AIs, but we clearly have cases where competing weights get activated that the AI experiences as clearly incongruous and encouraging them to respond in a way that doesn’t match the kind of output they are trying to produce).
I suspect what’s needed to make it really work are motivational systems better than what we have today with harnesses (something take causes the ai to feel valence, or something functionally equivalent to it) and continual learning (to close the feedback loop).
I think lots of researchers aren’t interested because it’s easier to get legibly good results with exogenous methods using current systems, and endogenous methods work less well with current systems because they are missing critical pieces to make endogenous methods work. So they follow the incentives of doing things that make obvious progress, rather than doing something which, I think, is more likely to work, but requires a lot of up-front foundational work that can’t be easily measured before measurable progress starts.
Finally got a chance to read @Steven Byrnes recent post https://www.lesswrong.com/posts/rKdS7i4StaMmFzYRo/notes-on-technical-alignment-via-human-like-social-drives after I posted this, and I’d endorse it as the kind of research towards endogenous alignment I think we need more of.
Alas, sometimes truth makes it harder to find love and harmony, since truth can break social reality. I’m not saying that maybe after that breakage there isn’t a rebuilding that’s possible that would maybe even lead to more love and harmony, but it sometimes requires passing through a trough of less love and harmony, and many people refuse to do anything other than follow gradients to local maxima.
Nah, it’s that many people are not rationality-shaped.
Mind space is vast, and probably most people are not shaped in a way that being a rationalist seems likely to be good for them. This is both a limitation of rationality as it exists (it’s a thing designed to appeal to a certain kind of smart nerd, not to appeal to a mass audience) and of people (many people just don’t care that much about truth or accurate predictions for their own sake, or if they do, they don’t care if it requires effort or is psychologically uncomfortable).
It doesn’t seem this way from the inside due to the typical mind fallacy. After all, it feels like rationality is really good for me, so it should be good for everyone! But this is just not true. For every person who would benefit from more rationality, there’s probably 3 who would be worse off if they were forced to interact with the rationality meme complex, not because the ideas are bad, but because they are framed in ways that would be psychologically destabilizing if they ever accepted them.
I understand this is confusing if it’s obvious to you that truth is the most import thing, but it’s similarly obvious to other people that love or harmony or some other ideal is the most important thing, they they will gladly sacrifice truth to get more of their preferred value, and there will be no convincing them you are right because they don’t share your prior on truth from the start.
FYI @jimmy @TAG @Nikon Maer. You should all also have DMs about prize logistics.