I’m wrote a book about why all knowledge of the truth is fundamentally uncertain. Give it a read!
I’ve also written a lot about AI safety, personal development, and Buddhism, among other things. Most of that’s here on LessWrong, but there’s also some on my blog, Uncertain Updates.
Boredom is the feeling that something more should be happening and isn’t and it tries to motivate us to do something. So the question becomes, why are you not okay just doing nothing or the not very exciting activity of what ever kind of meditation you are doing? It seems like that should be hedonically neutral, not negative, since while nothing especially good is happening, nothing especially bad is happening either other than just existing and perhaps noticing some aspect of existence.
For what is worth, this is a fairly common reaction to mediation, because mediation shows us the suffering that we’re dealing with all the time that we normally distract ourselves from (since aside from some rather exotic types of meditation, most methods don’t do anything other than bring your attention to what’s already experiencing). With the distractions removed, we see the suffering clearly, and don’t like it.
Thanks for writing this! I’m generally trying to avoid saying too much or responding too directly to any of the essays while the contest is open, but I wanted to express that I appreciate this does a much better job of getting at something I, probably inadequately, tried to address in the book in Chapter 9 in the section on debate, which is that fundamental uncertainty is rarely the basis on which to build an argument or decide anything.
For a certain kind of person (like myself) who’s deeply attracted to the idea of understanding everything, knowing about fundamental uncertainty is important to prevent hallucinating certainty where there is none, but beyond that fundamental uncertainty is rarely load bearing on any decisions, and in fact is a potential danger, as you note, if it stops thought. So I really appreciate you pushing on this point and exploring it in more detail.
I think the general point stands that “almost nobody” is working on alignment, but more people are working on alignment than you seem to think. What immediately came to mind that you left out was Softmax, and I also thought of teams internal to Anthropic and OpenAI that to some degree are working on alignment. Plus, “scattered people” is potentially quite valuable, as more people trying to work on this problem on their own increases the chances we get novel approaches that can be developed (if everyone is too concentrated they’ll tend to focus too much on similar ideas vs. trying new ideas).
Generally agree. I often try to redirect conversations about morality and ethics to conversations about norms. For a bunch of reasons people get confused using the morals abstraction, but the norms abstraction cuts through much of that confusion to ask “how do we expect people to behave” rather than “what’s good and bad”.
I think we should understand the meme version of Seeing Like a State et. al as a directional corrective. High modernism that totally ignores metis was bad in some ways and we backed off from it because of its excesses (e.g. scientific management is mostly dead), but ignoring the value of modernism would also give up the higher crop yields we now enjoy. The full version should be that it’s not either/or, but both/and, but it’s easy to see how that gets lost when a person just first realizes that we sometimes ignore metis too much.
With this kind of LLM behavior, I often try to ask the question “do humans do this?” and the answer in this case is yes.
Maybe not everyone does and not to this extent, but it’s not that uncommon for people to develop their own jargon, including ephemeral task specific jargon that is just relevant to the completion of a single task. It just looks a bit different when we do it, especially if we have to communicate using the jargon with others, where we take vague, normal words and give them temporary special meaning.
Can’t tell you how many times I was working on a programming problem and some vague words like “process” or “thing” or “item” came to take on special meaning that was nonobvious to anyone not clued in. What Mythos is doing seems pretty similar, but because it processes tokens/words differently than we do it looks different (what we do would probably look similarly alien to Mythos, except insofar as it’s trained to know that we do this).
I’m not sure what the take away from this observation is, but I think it’s worth making.
I walked away feeling like this is something of a nothing burger, simply because as best I can tell this is just publicly communicating what was already happening behind closed doors.
I’m surprised you don’t give more consideration to the possibility that the CCP will realize they can use AI to enforce party loyalty and automate ideological conformity and that AI might be able to do this better than humans, although maybe I misunderstand the true goals of the CCP as it currently exists.
As an author, I expect to be able to answer questions aimed to get me to say exactly what I mean! Don’t you?
I reject the framing that there is exactness about what I mean, per se. There is only what my words mean to me, and what they mean to others. I can at best say words that cause others to say words that I interpret to mean they understand what I mean, and sometimes this can be proven with high confidence by, say, using a mathematically precise model built on a agreed upon common grounding.
I’d instead say that I expect to answer questions aimed at clarifying confusion arising from differences of interpretation about what I intended to communicate, as well as questions about whether or not the world model induced in the reader’s mind (including my own mind reading my own words) by my words is accurate.
I don’t always get around to answering questions, but I would never, ever disparage someone asking real questions about my work as engaging in an “the vice of refusing to meaningfully engage” with me.
Have you never dealt with a troll whose trolling was disguised as real questions? Or more generally never dealt with someone who engaged in ways that exploited you for the purposes of their own agenda? My read is that you’re employing “real questions” to paper over a lot of complexity in how people comment on posts.
I’m not in a position to demand that they do it on my terms.
I mostly think you are, or at least are to the extent they choose to do it in the comments of something you wrote. For example, rather than a post on LessWrong, if I were hosting an academic symposium on nuclear physics, I’d reasonably be upset if a crank showed up to argue about perpetual motion. Similarly, I think there’s some good reason to push back if someone shows up and tries to yank control of the frame within the context of the comments on a post, especially if, like Said, they do it implicitly rather than explicitly.
I can’t think of a specific example, but for example I’d be much happier to see a comment that said “I think you’re wrong because you have the wrong framing. We should think about it this way...” than “You’re wrong because this contradicts this thing...” when the thing in question is a complex theory rather than some obvious, settled fact.
I agree that it’s fraught to try to answer the question of whose values to align to. In fact I think most serious alignment researchers agree on this point (alas there’s a bunch of less serious folks who’ve gotten caught up in trying to RL their way to alignment, which faces this problem of whose values to align to). And I agree there’s something to figure out about human psychology about what we’d consider flourishing and this is likely relatively fixed and not cultural specific, in that people from different cultures could learn to adapt to some kind of world that met certain features that induced whatever we think flourishing is.
But I get the impression you think some specific values are determinable in a scientific way, and that those can be aligned to? Maybe I’m misreading you and I need to wait for the next post?
What is required is a ground that is objective and can be found through reason in the natures of things rather than fixed by anyone’s say-so.
Why must the ground be “objective”? And furthermore, how do you intend to find something objective when no knowledge is properly objective (in the sense that word is usually used to mean)?
I’m not sure how to answer your question, because it’s not clear to me who’s world model you’re saying has facts A and B. In seems in the setup there’s some agent X who you propose can learn either A or B but not both, but these must not be initially in X’s world model else X would already know both.
Thanks for writing this. Appreciate your engagement with potential sources of errors. Since you submitted this for the contest, I’ll hold off on a substantive reply at this time, but I would like to press on one point I’d love to see you explore further. You say:
That being said, what we know is indeed contingent on the things to which we directed our efforts to make our world model more fine-grained. But it doesn’t mean, as the author states in the Thesis, that “the truth that can be known is not independent of us, but rather dependent on that for which we care.”
My reading is that you meant for this to be an argument against the thesis, but as I read it, it’s not really making such an argument, just stating that a statement of your own doesn’t support the one I made in the book. I’d be very interested to see your full argument against this part of the thesis, assuming I’m correct that you disagree with it. I can’t promise I’ll agree with whatever argument you’d make, but I’d be very much interested to see the argument!
I think the karma system works fine for genuine Gricean errors. If someone asks a dumb question that isn’t worth your time, you can downvote it and move on with your day, trusting that readers will also recognize it as a dumb question and not think less of you for not answering it. The desire to purge users who ask annoying questions (for imposing a cost on authors who are afraid of looking bad if they don’t reply) would seem to rest on an implicit recognition that readers don’t think the annoying questions are dumb Gricean failures.
In practice I don’t recall many cases of people downvoting Gricean errors. And in particular Said was really good at pointing out Gricean errors relative to the framing presented in the Sequences. That would be fine, except he’d do it when a post was specifically trying to reject some part of the framing provided by the Sequences, which is not really engaging with the post so much as attempting to say “i reject your framing” without actually arguing for why the framing is rejected.
(My general complaint is that Said doesn’t argue in good faith or seem to know what that means. What you see as his virtue of asking questions to get you to say exactly what you mean, I see as the vice of refusing to meaningfully engage with the author to try to understand them.)
Partial government ownership of AI labs seems likely to happen one way or another, simply because I think military leadership already understands correctly that AI labs will hold the potential for greater military power than the government does, and they’ll want tighter control. They already have this any number of ways with companies that produce war materiel, so I expect them to demand it of AI labs, too.
Yes. I specifically have worse autobiographical memory. This seems to happen because I’m less narrating my life and less reliving recent moments due to reduced neuroticism. So a lot more of my life just happens without converting into episodic memory.
It continues to be very interesting to see LessWrong wake up to politics as politics wakes up to LessWrong.