I’m wrote a book about why all knowledge of the truth is fundamentally uncertain. Give it a read!
I’ve also written a lot about AI safety, personal development, and Buddhism, among other things. Most of that’s here on LessWrong, but there’s also some on my blog, Uncertain Updates.
I’m surprised you don’t give more consideration to the possibility that the CCP will realize they can use AI to enforce party loyalty and automate ideological conformity and that AI might be able to do this better than humans, although maybe I misunderstand the true goals of the CCP as it currently exists.
As an author, I expect to be able to answer questions aimed to get me to say exactly what I mean! Don’t you?
I reject the framing that there is exactness about what I mean, per se. There is only what my words mean to me, and what they mean to others. I can at best say words that cause others to say words that I interpret to mean they understand what I mean, and sometimes this can be proven with high confidence by, say, using a mathematically precise model built on a agreed upon common grounding.
I’d instead say that I expect to answer questions aimed at clarifying confusion arising from differences of interpretation about what I intended to communicate, as well as questions about whether or not the world model induced in the reader’s mind (including my own mind reading my own words) by my words is accurate.
I don’t always get around to answering questions, but I would never, ever disparage someone asking real questions about my work as engaging in an “the vice of refusing to meaningfully engage” with me.
Have you never dealt with a troll whose trolling was disguised as real questions? Or more generally never dealt with someone who engaged in ways that exploited you for the purposes of their own agenda? My read is that you’re employing “real questions” to paper over a lot of complexity in how people comment on posts.
I’m not in a position to demand that they do it on my terms.
I mostly think you are, or at least are to the extent they choose to do it in the comments of something you wrote. For example, rather than a post on LessWrong, if I were hosting an academic symposium on nuclear physics, I’d reasonably be upset if a crank showed up to argue about perpetual motion. Similarly, I think there’s some good reason to push back if someone shows up and tries to yank control of the frame within the context of the comments on a post, especially if, like Said, they do it implicitly rather than explicitly.
I can’t think of a specific example, but for example I’d be much happier to see a comment that said “I think you’re wrong because you have the wrong framing. We should think about it this way...” than “You’re wrong because this contradicts this thing...” when the thing in question is a complex theory rather than some obvious, settled fact.
I agree that it’s fraught to try to answer the question of whose values to align to. In fact I think most serious alignment researchers agree on this point (alas there’s a bunch of less serious folks who’ve gotten caught up in trying to RL their way to alignment, which faces this problem of whose values to align to). And I agree there’s something to figure out about human psychology about what we’d consider flourishing and this is likely relatively fixed and not cultural specific, in that people from different cultures could learn to adapt to some kind of world that met certain features that induced whatever we think flourishing is.
But I get the impression you think some specific values are determinable in a scientific way, and that those can be aligned to? Maybe I’m misreading you and I need to wait for the next post?
What is required is a ground that is objective and can be found through reason in the natures of things rather than fixed by anyone’s say-so.
Why must the ground be “objective”? And furthermore, how do you intend to find something objective when no knowledge is properly objective (in the sense that word is usually used to mean)?
I’m not sure how to answer your question, because it’s not clear to me who’s world model you’re saying has facts A and B. In seems in the setup there’s some agent X who you propose can learn either A or B but not both, but these must not be initially in X’s world model else X would already know both.
Thanks for writing this. Appreciate your engagement with potential sources of errors. Since you submitted this for the contest, I’ll hold off on a substantive reply at this time, but I would like to press on one point I’d love to see you explore further. You say:
That being said, what we know is indeed contingent on the things to which we directed our efforts to make our world model more fine-grained. But it doesn’t mean, as the author states in the Thesis, that “the truth that can be known is not independent of us, but rather dependent on that for which we care.”
My reading is that you meant for this to be an argument against the thesis, but as I read it, it’s not really making such an argument, just stating that a statement of your own doesn’t support the one I made in the book. I’d be very interested to see your full argument against this part of the thesis, assuming I’m correct that you disagree with it. I can’t promise I’ll agree with whatever argument you’d make, but I’d be very much interested to see the argument!
I think the karma system works fine for genuine Gricean errors. If someone asks a dumb question that isn’t worth your time, you can downvote it and move on with your day, trusting that readers will also recognize it as a dumb question and not think less of you for not answering it. The desire to purge users who ask annoying questions (for imposing a cost on authors who are afraid of looking bad if they don’t reply) would seem to rest on an implicit recognition that readers don’t think the annoying questions are dumb Gricean failures.
In practice I don’t recall many cases of people downvoting Gricean errors. And in particular Said was really good at pointing out Gricean errors relative to the framing presented in the Sequences. That would be fine, except he’d do it when a post was specifically trying to reject some part of the framing provided by the Sequences, which is not really engaging with the post so much as attempting to say “i reject your framing” without actually arguing for why the framing is rejected.
(My general complaint is that Said doesn’t argue in good faith or seem to know what that means. What you see as his virtue of asking questions to get you to say exactly what you mean, I see as the vice of refusing to meaningfully engage with the author to try to understand them.)
Partial government ownership of AI labs seems likely to happen one way or another, simply because I think military leadership already understands correctly that AI labs will hold the potential for greater military power than the government does, and they’ll want tighter control. They already have this any number of ways with companies that produce war materiel, so I expect them to demand it of AI labs, too.
Yes. I specifically have worse autobiographical memory. This seems to happen because I’m less narrating my life and less reliving recent moments due to reduced neuroticism. So a lot more of my life just happens without converting into episodic memory.
I view this post through the lens that most bad habits are things like smoking and drinking. Done in sufficient moderation, they are fine, but done too much, and they are destructive.
For example, let’s say Alice likes smoking, but know that if she smokes a pack a day it’ll be bad for her health. She does some math that decide that 1 cigarette a week is as much health risk as she can afford for the joy of smoking. Or, to make the implementation simpler, let’s say 3 packs a year (60 cigarettes per annum).
It’s not unreasonable that she might do something like buy 3 packs at the start of the year, and when she’s out she’s out. It’s a simple commitment to say she will buy no more packs, but then give herself freedom to smoke those packs however she wants. It contains the damage, minimizes the risk, and eliminates the difficulty of having zero cigarettes.
Similar solutions can apply to most habits you consider bad: give yourself a bad habit budget, spend the budget how you want, and hold yourself to the budget.
Yes, they are at times agentic, but we build agents out of LLMs.
The LLMness is mostly found down in the Slow Zone. In the Transcend, the Powers are relatively inscrutable and Vinge avoids saying what they’re up to, other than the bits and pieces of action that we see from them.
I could tell you, but perhaps better to be forever uncertain about it!
Accompanying an instruction with the why significantly improves
Thanks for this suggestion. I’ve also found that including examples is less useful in many cases with 4.7 relative to previous models, in that it can anchor too strongly on the examples and fail to be flexible, which is basically the opposite problem of earlier models, where they were so fuzzy that even with examples you got some degree of flexibility for free.
Sure, within a fixed framework, we get convergence, even when using different methods. But this is possible because concepts like “the radius of the chord” are already agreed on. The paradox isn’t about not knowing which method generated the chord, it’s that the question itself is underspecified and can’t be fixed without making a choice (introducing contingency) because we lack access to a view from nowhere to adjudicate it.
But the real answer to Bertrand’s paradox is that the word ‘random’ is not meaningful without any knowledge of the structure of your sampling process.
Indeed, this is a fully general feature of attempting to know what’s true. What we’d like to do is create an objective map the territory, but any map we construct is contingent on large numbers of non-objective assumptions that hinge on everything from sampling, selection, and attention processes to what categories have been historically useful to include in our ontology. This limitation stands in the way of many clever arguments that would seek to do an end-run around uncertainty, but ultimately fail because they smuggle in unjustified assumptions (cf. anthropic arguments).
The answer lies not in fear, but in shame and embarrassment.
Your friend probably wasn’t actually scared of scaring the Uber driver. They were likely instead scared of having to defend their views to the Uber driver. They understood that taking COVID seriously was not yet a defensible, mainstream view, and could feel the potential shame and embarrassment the driver would have directed towards them for being scared of something that they don’t believe is real.
It’s the same reason people try to avoid talking about their wacky conspiracy theory beliefs in polite company. They know doing that is low status, and people who see status can feel their status lowering as they say the words “9/11 was an inside job” or “the moon landing was faked”. That’s also why, until quite recently, talking about AI risks was so hard: it was low status to do so. The only thing that changed is that lots of high status people started to agree there are big AI risks, so now we can all safely talk about AI risks without losing status.
Perhaps so. Maybe I can entice you to write something longer on this point, as I just opened an essay contest for the book: https://www.lesswrong.com/posts/vMJbnnvvrHNw728NC/fundamental-uncertainty-usd2-000-essay-contest
I walked away feeling like this is something of a nothing burger, simply because as best I can tell this is just publicly communicating what was already happening behind closed doors.