gb

Karma: 45

gb 9 Oct 2024 16:17 UTC
2 points
1
in reply to: Nathan Young’s comment on: Advice for journalists
I also don’t think privacy is a binary.
That’s an interesting perspective. I could subscribe to the idea that journalists may be missing the optimal point there, but that feels a bit weaker than your initial assertion.
Do you think that a conversation we have in LessWrong dms is as public as if I tweeted it?
I mean, I would not quote a DM without asking first. But I understand that as a kind of charity, not an ethical obligation, and while I try my best to be charitable towards others, I do not expect (nor do I feel in any way entitled to) the same level of compassion.

gb 8 Oct 2024 23:18 UTC
1 point
−1
in reply to: Brendan Long’s comment on: Advice for journalists
There’s definitely a fair expectation against gossiping and bad-mouthing. I don’t think that’s quite what the OP is talking about, though. I believe the relevant distinction is that (generally speaking) those behaviors don’t do any good to anyone, including the person spreading the gossip. But consider how murkier the situation becomes if you’re competing for a promotion with the person here:
if you overheard someone saying something negative about their job and then going out of your way to tell their boss.

gb 8 Oct 2024 21:35 UTC
1 point
−3
in reply to: Brendan Long’s comment on: Advice for journalists
My understanding is that the OP is suggesting the journalists’ attitude is unreasonable (maybe even unethical). You’re saying that their attitude is justifiable because it benefits their readers. I don’t quite agree that that reason is necessary, nor that it would be by itself sufficient. My view is that journalists are justified in quoting a source because anyone is generally justified in quoting what anyone else has actually said, including for reasons that may benefit no one but the quoter. There are certainly exceptions to this (if divulging the information puts someone in danger, for instance), but those really are exceptions, not the rule. The rule, as recognized both by common practice and by law, is that you simply have no general right to (or even expectation of) privacy about things you say to strangers, unless of course the parties involved agree otherwise.

gb 7 Oct 2024 19:16 UTC
0 points
−1
in reply to: green_leaf’s comment on: Alignment by default: the simulation hypothesis
This sounds absurd to me. Unless of course you’re taking the “two golden bricks” literally, in which case I invite you to substitute it by “saving 1 billion other lives” and seeing if your position still stands.

gb 7 Oct 2024 18:59 UTC
5 points
−7
on: Advice for journalists
I didn’t downvote, but I would’ve hard disagreed on the “privacy” part if only there were a button for that. It’s of course a different story if they’re misquoting you, or taking quotes deliberately out of context to mislead. But to quote something you actually said but on second thought would prefer to keep out of publication is… really kind of what journalists need to do to keep people minimally well-informed. Your counterexamples involve communications with family and friends, and it’s not very clear to me why the same heuristic should be automatically applied to conversations with strangers. But in any case, not even with the former your communication is “truly” private, as outside of very narrow exceptions like marital privilege, their testimony (on the record, for potentially thousands of people to read too) may be generally compelled under threat of arrest.

gb 6 Oct 2024 18:03 UTC
0 points
−1
in reply to: green_leaf’s comment on: Alignment by default: the simulation hypothesis
The problem here is that the set of all possible commands for which I can’t (by that definition) be maximally rewarded is so vast that the statement “if someone maximally rewards/punishes you, their orders are your purpose of life” becomes meaningless.
Not true, as the reward could include all of the unwanted consequences of following the command being divinely reverted a fraction of a second later.

gb 4 Oct 2024 16:55 UTC
1 point
0
in reply to: faul_sname’s comment on: Alignment by default: the simulation hypothesis
That’s a great question. If it turns out to be something like an LLM, I’d say probably yes. More generally, it seems to me at least plausible that a system capable enough to take over would also (necessarily or by default) be capable of abstract reasoning like this, but I recognize the opposite view is also plausible, so the honest answer is that I don’t know. But even if it is the latter, it seems that whether or not the system would have such abstract-reasoning capability is something at least partially within our control, as it’s likely highly dependent on the underlying technology and training.

gb 4 Oct 2024 15:28 UTC
−2 points
−3
in reply to: green_leaf’s comment on: Alignment by default: the simulation hypothesis
To be rewarded (and even more so “maximally rewarded”) is to be given something you actually want (and the reverse for being punished). That’s the definition of what a reward/punishment is. You don’t “choose” to want/not want it, any more than you “choose” your utility function. It just is what it is. Being “rewarded” with something you don’t want is a contradiction in terms: at best someone tried to reward you, but that attempt failed.

gb 4 Oct 2024 13:42 UTC
−2 points
−3
in reply to: Ape in the coat’s comment on: Alignment by default: the simulation hypothesis

Not at all. You still have to evaluate this offer using your own mind and values. You can’t sidestep this process by simply assuming that Creator’s will by definition is the purpose of your life, and therefore you have no choice but to obey.

I’ll focus on this first, as it seems that the other points would be moot if we can’t even agree on this one. Are you really saying that even if you know with 100% certainty that God exists AND lays down explicit laws for you to follow AND maximally rewards you for all eternity for following those laws AND maximally punishes you for all eternity for failing to folllow those laws, you would still have to “evaluate” and could potentially arrive at a conclusion other than that the purpose of life is follow God’s laws?

gb 4 Oct 2024 13:32 UTC
1 point
0
in reply to: Seth Herd’s comment on: Alignment by default: the simulation hypothesis

Why would humans be testing AGIs this way if they have the resources to create simulation that will fool a super intelligence?

My argument is more that the ASI will be “fooled” by default, really. It might not even need to be a particularly good simulation, because the ASI will probably not even look at it before pre-commiting not to update down on the prior of it being a simulation.

But to answer your question, possibly because it might be the best way to test for alignment. We can create an AI that generates realistic simulations, and use those to test other ASIs.

Also, the risk of humanity being wiped out seems different and worse while that asi is attempting a takeover—during that time the humans are probably an actual threat.

Downstream of the above.

Finally, leaving humans around would seem to pose a nontrivial risk that they’ll eventually spawn a new ASI that could threaten the original.

The Dyson sphere is just a tiny part of the universe so using that as the fractional cost seems wrong. Other considerations in both directions would seem to dominate it.

We can be spared and yet not allowed to build further ASIs. The cost of enforcing such restriction is negligible compared to the loss of output due to the hole in the Dyson sphere.

gb 3 Oct 2024 10:25 UTC
−2 points
−3
in reply to: Ape in the coat’s comment on: Alignment by default: the simulation hypothesis

Otherwise, it would mean that it’s only possible to create simulations where everyone is created the same way as in the real world.

It’s certainly possible for simulations to differ from reality, but they seem less useful the more divergent from reality they are. Maybe the simulation could be for pure entertainment (more like a video game), but you should ascribe a relatively low prior to that IMO.

The discussion of theism vs atheis is about the existence of God. Obviously if we knew that God exists the discussion would evaporate. However the question of purpose of life would not.

There’s a reason people don’t have the same level of enthusiasm when discussing the existence of dragons, though. If dragons do exist, that changes nothing: you’d take it as a curiosity and move on with your life. Certainly not so if you were to conclude that God exists. Maybe you can still not know with 100% certainty what it is that God wants, but can we at least agree it changes the distribution of probabilities somehow?

Even if I can infer the desires of my creator, this doesn’t bridge the is-ought gap and doesn’t make such desires the objective purpose of my life. I’ll still have to choose whether to satisfy these desires or not.

It does if you simultaneously think your creator will eternally reward you for doing so, and/or eternally punish you for failing to. Which if anything seems even more obvious in the case of a simulation, btw.

gb 2 Oct 2024 20:32 UTC
0 points
−3
in reply to: Ape in the coat’s comment on: Alignment by default: the simulation hypothesis
I’m afraid your argument proves too much. By that exact same logic, knowing you were created by a more powerful being (God) would similarly tell you absolutely nothing about what the purpose of life is, for instance. If that were true, the entire discussion of theism vs. atheism would suddenly evaporate.

gb 30 Sep 2024 19:01 UTC
1 point
0
in reply to: Ape in the coat’s comment on: Alignment by default: the simulation hypothesis
Thinking about this a bit more, I realize I’m confused.
Aren’t you arguing that AI will be aligned by default?
I really thought I wasn’t before, but now I feel it would only require a simple tweak to the original argument (which might then be proving too much, but I’m interested in exploring more in depth what’s wrong with it).
Revised argument: there is at least one very plausible scenario (described in the OP) in which the ASI is being simulated precisely for its willingness to spare us. It’s very implausible that it would be simulated for the exact opposite goal, so us not getting spared is, in all but the tiniest subset of cases, an unintended byproduct. Since that byproduct is avoidable with minimal sacrifice of output (of the order of 4.5e-10), it might as well be avoided just in case, given I expect the likelihood of the simulation being run for the purpose described in the OP to be a few orders of magnitude higher, as I noted earlier.
I don’t quite see what’s wrong with this revised argument, save for the fact that it seems to prove too much and that other people would probably already have thought of it if it were true. Why isn’t it true?

gb 30 Sep 2024 16:10 UTC
0 points
−1
in reply to: Ape in the coat’s comment on: Alignment by default: the simulation hypothesis
I think you’re interpreting far too literally the names of the simulation scenarios I jotted down. Your ability to trade is compromised if there’s no one left to trade with, for instance. But none of that matters much, really, as those are meant to be illustrative only.
Aren’t you arguing that AI will be aligned by default?
No. I’m really arguing that we don’t know whether or not it’ll be aligned by default.
As there is no particular reason to expect that it’s the case,
I also don’t see any particular reason to expect that the opposite would be the case, which is why I maintain that we don’t know. But as I understand it, you seem to think there is indeed reason to expect the opposite, because:
Sadly for us, survival of humanity is a very specific thing. This is just the whole premise of the alignment problem once again.
I think the problem here is that is that you’re using the word “specific” with a different meaning than people normally use in this context. Survival of humanity sure is a “specific” thing in the sense that it’ll require specific planning on the part of the ASI. It is however not “specific” in the sense that it’s hard to do if the ASI wants it done, it’s just that we don’t know how to make it want that. Abstract considerations about simulations might just do the trick automatically.

gb 30 Sep 2024 12:58 UTC
1 point
0
in reply to: Ape in the coat’s comment on: Alignment by default: the simulation hypothesis
Or it could be:

SimulatedAndBeingTestedForAchievingGoalsWithoutBeingNoticed

SimulatedAndBeingTestedForAbilityToTradeWithCreators

SimulatedAndBeingTestedForWillignessToSitQuietAndDoNothing

…

SimulatedAndBeingTestedForAnyXThatDoesNotLeadToDeathOfCreators

…

None of the things here nor in your last reply seems particularly likely, so there’s no telling in principle which set outweighs the other. Hence my previous assertion that we should be approximately completely unsure of what happens.

gb 30 Sep 2024 3:59 UTC
1 point
0
in reply to: habryka’s comment on: You can, in fact, bamboozle an unaligned AI into sparing your life
I was writing a reply and realized I can make the argument even better. Here’s a sketch. If our chances of solving the alignment problem are high, the AI will think it’s likely to be in a simulation (and act accordingly) regardless of any commitments by us to run such simulations in the future – it’ll just be a plausible explanation of why all those intelligent beings that should likely have solved the alignment problem seemingly did not in the reality the AI is observing. So we can simply ask the hypothetical aligned AI, after it’s created, what were our odds of solving the alignment problem in the first place (just to make sure that us solving it wasn’t a cosmological strike of luck), and spare the cost of running simulations. Hence simulations of the kind the OP is describing would be run primarily in the subset of worlds in which we indeed solve the alignment problem by a strike of luck. We can thus balance this in such a way that the likelihood of the AI being in a simulation is virtually independent of the likelihood of us solving the alignment problem!

gb 30 Sep 2024 3:02 UTC
1 point
−2
in reply to: habryka’s comment on: You can, in fact, bamboozle an unaligned AI into sparing your life
their force of course depends on the degree to which you think alignment is easy or hard.
I don’t think that’s true. Even if the alignment problem is hard enough that the AI can be ~100% sure humans would never solve it, reaching such conclusion would require gathering evidence. At the very least, it would require evidence of how intelligent humans are – in other words, it’s not something the AI could possibly know a priori. And so passing the simulation would presumably require pre-commiting to spare humans before gathering such evidence.

gb 29 Sep 2024 16:55 UTC
1 point
0
in reply to: Garrett Baker’s comment on: Any Trump Supporters Want to Dialogue?
A steelman is not necessarily an ITT, but whenever you find yourself having “0% support” for a position ~half the population supports, it’s almost guaranteed that the ITT will be a steelman of your current understanding of the position.

gb 29 Sep 2024 14:32 UTC
7 points
5
in reply to: Pazzaz’s comment on: Any Trump Supporters Want to Dialogue?
I highly doubt anywhere near the majority of Trump supporters (or even Trump himself) give any credence to the literal truth of those claims. It’s much more likely that they simply don’t care whether it’s literally true or not, because they feel that the “underlying” is true or something of the kind. When it comes to hearsay, people are much more forgiving of literal falsehoods, especially when they acknowledge there is a kind of “metatruth” to it. To give an easy analogue, of all the criticism I’ve heard of Christianity, not once have I heard anyone complain that the parables told by Jesus weren’t literally true, for example. (I do believe my account here passes the IIT for both groups, btw.)

gb 26 Sep 2024 22:55 UTC
1 point
0
in reply to: Ape in the coat’s comment on: Alignment by default: the simulation hypothesis
Sure. But I think you’re reading my argument to be stronger than I mean it to be. Which is partially my fault since I made my previous replies a bit too short, and for that I apologize.

What I’m doing here is presenting one particular simulation scenario that (to me) seems quite plausible within the realm of simulations. I’m not claiming that that one scenario dominates all others combined. But luckily that stronger claim is really not necessary to argue against Eliezer’s point: the weaker one suffices. Indeed, if the scenario I’m presenting is more than 4.5e-10 likely (and I do think it’s much more likely than that, probably by a few orders of magnitude), than it is more than enough to outweigh the practical cost of the ASI having to build a Dyson shell with a hole with the order of 4.5e-10 of it’s surface area.

Now, that scenario is (I claim) the most likely one, conditional of course on a simulation taking place to begin with. The other candidate simulation scenarios are various, and none of them seems particularly likely, though combined they might well outweigh this one in terms of mass probability, as I already acknowledged. But so what? Are you really claiming that the distribution of those other simulation scenarios is skewed enough to tilt the scales back to the doom side? It might be, but that’s a much harder argument to make. I’m approximately completely unsure, which seems way better than the 99%+ chance Eliezer seems to give to total doom. So I guess I’d count that as good news.