So far a journalist just said “sure”. So n = 1 it’s fine.
Nathan Young
Karma: 1,422
Trying out my new journalist strategy.
Did you reformat all the footnotes or do you have a tool for that?
My main takeaway from this series is that Carlsmith seems to be gesturing at some important things where I want a more diagrammy, mathsy approach to come along after.
What does “Green” look like in more blue terms? When specifically might we want to be paperclippers and when not? Where are the edges of the different concepts?
So by my metric, Yudkowsky and Lintemandain’s Dath Ilan isn’t neutral, it’s quite clearly lawful good, or attempting to be. And yet they care a lot about the laws of cognition.
So it seems to me that the laws of cognition can (should?) drive towards flouishing rather than pure knowledge increase. There might be things that we wish we didn’t know for a bit. And ways to increase our strength to heal rather than our strength to harm.
To me it seems a better rationality would be lawful good.
Yeah I find the intention vs outcome thing difficult.
What do you think of “average expected value across small perturbations in your life”. Like if you accidentally hit churchill with a car and so cause the UK to lose WW2 that feels notably less bad than deliberately trying to kill a much smaller number of people. In many nearby universes, you didn’t kill churchill, but in many nearby universes that person did kill all those people.
Here is a 5 minute, spicy take of an alignment chart.
What do you disagree with.
To try and preempt some questions:
Why is rationalism neutral?
It seems pretty plausible to me that if AI is bad, then rationalism did a lot to educate and spur on AI development. Sorry folks.
Why are e/accs and EAs in the same group.
In the quick moments I took to make this, I found both EA and E/acc pretty hard to predict and pretty uncertain in overall impact across some range of forecasts.
Under considered might be more accurate?
And yes, I agree that seems bad.
Joe Rogan (largest podcaster in the world) giving repeated concerned mediocre x-risk explanations suggests that people who have contacts with him should try and get someone on the show to talk about it.
eg listen from 2:40:00 Though there were several bits like this during the show.
Weakly endorsed
“Curiously enough, the only thing that went through the mind of the bowl of petunias as it fell was Oh no, not again. Many people have speculated that if we knew exactly why the bowl of petunias had thought that we would know a lot more about the nature of the Universe than we do now.”
The Hitchhiker’s Guide To The Galaxy, Douglas Adams
Feels like FLI is a massively underrated org. Cos of the whole vitalik donation thing they have like $300mn.
Re safety, I don’t know about Oakland but some parts of SF are genuinely the most dangerous feeling places I’ve ever been to after dark (because normally I wouldn’t go somewhere, but SF feels very fine until it isn’t). If I am travelling to places in SF after dark I’ll check how dodgy the street entrances are.
What are the LessWrong norms on promotion? Writing a post about my company seems off (but I think it could be useful to users). Should I write a quick take?
AI and integrity
Given my understanding of epistemic and necessary truths it seems plausible that I can construct epistemic truths using only necessary ones, which feels contradictory.
Eg 1 + 1 = 2 is a necessary truth
But 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 = 10 is epistemic. It could very easily be wrong if I have miscounted the number of 1s.
This seems to suggest that necessary truths are just “simple to check” and that sufficiently complex necessary truths become epistemic because of a failure to check an operation.
Similarly “there are 180 degrees on the inside of a triangle” is only necessarily true in spaces such as R2. It might look necessarily true everywhere but it’s not on the sphere. So what looks like a necessary truth actually an epistemic one.
What am I getting wrong?
A problem with overly kind PR is that many people know that you don’t deserve the reputation. So if you start to fall, you can fall hard and fast.
Likewise it incentivises investigation that you can’t back up.
If everyone thinks I am lovely, but I am two faced, I create a juicy story any time I am cruel. Not so if am known to be grumpy.
eg My sense is that EA did this a bit with the press tour around What We Owe The Future. It built up a sense of wisdom that wasn’t necessarily deserved, so with FTX it all came crashing down.
Personally I don’t want you to think I am kind and wonderful. I am often thoughtless and grumpy. I think you should expect a mediocre to good experience. But I’m not Santa Claus.
I am never sure whether rats are very wise or very naïve to push for reputation over PR, but I think it’s much more sustainable.
@ESYudkowsky can’t really take a fall for being goofy. He’s always been goofy—it was priced in.
Many organisations think they are above maintaining the virtues they profess to possess, instead managing it with media relations.
In doing this they often fall harder eventually. Worse, they lose out on the feedback from their peers accurately seeing their current state.
Journalists often frustrate me as a group, but they aren’t dumb. Whatever they think is worth writing, they probably have a deeper sense of what is going on.
Personally I’d prefer to get that in small sips, such that I can grow, than to have to drain my cup to the bottom.
I’ve made a big set of expert opinions on AI and my inferred percentages from them. I guess that some people will disagree with them.
I’d appreciate hearing your criticisms so I can improve them or fill in entries I’m missing.
Though sometimes the obligation to answer is right, right? I guess maybe it’s that obligation works well at some scale, but then becomes bad at some larger scale. In a coversation, it’s fine, in a public debate, sometimes it seems to me that it doesn’t work.
I think the motivating instances are largely:
Online debates are bad
Freedom Of Information requests suck
I think I probably backfilled from there.
I do sometimes get persistant questions on twitter, but I don’t think there is a single strong example.
Thanks to the people who use this forum.
I try and think about things better and it’s great to have people to do so with, flawed as we are. In particularly @KatjaGrace and @Ben Pace.
I hope we can figure it all out.