bogdanb

Karma: 1,225

bogdanb 21 Jul 2023 7:02 UTC
2 points
0
in reply to: TekhneMakre’s comment on: Where I agree and disagree with Eliezer
I’m not sure I understand your weighting argument. Some capabilities are “convergently instrumental” because they are useful for achieving a lot of purposes. I agree that AIs construction techniques will target obtaining such capabilities, precisely because they are useful.
But if you gain a certain convergently instrumental capability, it then automatically allows you to do a lot of random stuff. That’s what the words mean. And most of that random stuff will not be safe.
I don’t get what the difference is between “the AI will get convergently instrumental capabilities, and we’ll point those at AI alignment” and “the AI will get very powerful and we’ll just ask it to be aligned”, other than a bit of technical jargon.
As soon as the AI it gets sufficiently powerful [convergently instrumental capabilities], it is already dangerous. You need to point it precisely at a safe target in outcomes-space or you’re in trouble. Just vaguely pointing it “towards AI alignment” is almost certainly not enough; specifying that outcome safely is the problem we started with.
(And you still have the problem that while it’s working on that someone else can point it at something much worse.)

bogdanb 2 Jul 2022 20:56 UTC
3 points
1
in reply to: johnlawrenceaspden’s comment on: Where I agree and disagree with Eliezer
Exactly. You can’t generalize from “natural” examples to adversarial examples. If someone is trying hard to lie to you about something, verifying what they say can very well be harder than finding the truth would have been absent their input, particularly when you don’t know if and what they want to lie about.
I’m not an expert in any of these and I’d welcome correction, but I’d expect verification to be at least as hard as “doing the thing yourself” in cases like espionage, hacking, fraud and corruption.

bogdanb 2 Jul 2022 20:24 UTC
2 points
0
in reply to: TekhneMakre’s comment on: Where I agree and disagree with Eliezer
AI accelerates the timetable for things we know how to point AI at
It also accelerates the timetable for random things that we don’t expect and don’t even try to point the AI at but that just happen to be easier for incrementally-better AI to do.
Since the space of stuff that helps alignment seems much smaller than the space of dangerous things, you’d expect most things the AI randomly accelerates without us pointing it at will be dangerous.

bogdanb 21 Jun 2022 10:04 UTC
4 points
in reply to: Lone Pine’s comment on: Parable: The Bomb that doesn’t Explode
See above. Don’t become a munitions engineer, and, being aware that someone else will take that role, try to prevent anyone from taking that role. (Hint: That last part is very hard.)
The conclusions might change if planet-destroying bombs are necessary for some good reason, or if you have the option of safely leaving the planet and making sure nobody that comes with you will also want to build planet-destroying bombs. (Hint: That last part is still hard.)

bogdanb 2 Jun 2015 21:14 UTC
0 points
in reply to: [deleted]’s comment on: Bragging Thread May 2015
For what it’s worth, the grammar and spelling was much better than is usual for even the native English part of the Internet. That’s probably fainter praise than it deserves, I don’t remember actually noticing any such fault, which probably means there are few of them.

The phrasing and wording did sound weird, but I guess that’s at least one reason why you’re writing, so congratulations and I hope you keep it up! I’m quite curious to see where you’ll take it.

bogdanb 28 Feb 2015 23:19 UTC
3 points
in reply to: tohu’s comment on: Harry Potter and the Methods of Rationality discussion thread, February 2015, chapter 113
Indeed, the only obvious “power” Harry has that is (as far as we know) unique to him is Partial Transfiguration. I’m not sure if Voldie “knows it not”; as someone mentioned last chapter, Harry used it to cut trees when he had his angry outburst in the Forbidden Forest, and in Azkhaban as well. In the first case Voldie was nearby, allegedly to watch out for Harry, but far enough that to be undetectable via their bond, so it’s possible he didn’t see what exact technique Harry used. In Azkhaban as well he was allegedly unconscious.

I can’t tell if he could have deduced the technique only by examining the results. (At least for the forest occasion he could have made time to examine the scene carefully, and I imagine that given the circumstances he’d have been very interested to look into anything unusual Harry seemed to be able to do.)

On the plus side, Harry performed PT by essentially knowing that objects don’t exist; so it could well be possible to transfigure a thin slice of thread of air into something strong enough to cut. For that matter, that “illusion of objects” thing should allow a sort of “reverse-Partial” transfiguration, i.e. transfigure (parts of) many objects into a single thing. Sort of like what he did to the troll’s head, but applied simultaneously to a slice of air, wands, and Death Eaters. Dumbledore explicitly considers it as a candidate against Voldemort (hint, Minerva remembers Dumbledore using transfiguration in combat). And, interestingly, it’s a wordless spell (I’m not even sure if Harry can cast anything* else wordlessly), and Harry wouldn’t need to raise his wand, or even move at all, to cast it on air (or on the time-space continuum, or world wave-function, whatever).

On the minus side, I’m not sure if he could do it fast enough to kill the Death Eaters before he’s stopped. He did get lots of transfiguration training, and using it in anger in the forest suggests he can do it pretty fast, but he is watched, and IIRC transfiguration is not instantaneous. He probably can’t cast it on Voldie nor on his wand, though he might be able to destroy the gun. And Voldemort can certainly find lots of ways to kill him without magic or touching him directly; hell, he probably knows kung fu and such. And even if Harry managed to kill this body, he’d have to find a way to get rid of the Horcruxes. (I still don’t understand exactly what the deal is with those. Would breaking the Resurrection Stone help?)

bogdanb 17 Feb 2015 22:18 UTC
0 points
in reply to: Ben Pace’s comment on: Harry Potter and the Methods of Rationality discussion thread, February 2015, chapter 104
Well, we only know that Harry feels doom when near Q and/or his magic, and that in one case in Azkhaban something weird happened when Harry’s Patronus interacted with what appeared to be an Avada Kedavra bolt, and that Q appears to avoid touching Harry.

Normally I’d say that faking the doom sensations for a year, and faking being incapacitated while trying to break someone out of Azkhaban, would be too complicated. But in this case...

bogdanb 17 Feb 2015 20:50 UTC
0 points
in reply to: VAuroch’s comment on: The Great Filter is early, or AI is hard
Both good points, thank you.

bogdanb 17 Feb 2015 20:38 UTC
0 points
in reply to: TylerJay’s comment on: The Great Filter is early, or AI is hard
Thank you, that was very interesting!

bogdanb 5 Sep 2014 23:57 UTC
0 points
in reply to: Richard_Kennaway’s comment on: Truth vs Utility
I sort of get your point, but I’m curious: can you imagine learning (with thought-experiment certainty) that there is actually no reality at all, in the sense that no matter where you live, it’s simulated by some “parent reality” (which in turn is simulated, etc., ad infinitum)? Would that change your preference?

bogdanb 5 Sep 2014 23:23 UTC
2 points
in reply to: TylerJay’s comment on: The Great Filter is early, or AI is hard

most “earthlike” planets in habitable zones around sunlike stars are on average 1.8 Billion years older than the Earth

How do you know? (Not rhethorical, I have no idea and I’m curious.)

bogdanb 5 Sep 2014 23:19 UTC
1 point
in reply to: VAuroch’s comment on: The Great Filter is early, or AI is hard
If the final goal is of local scope, energy acquisition from out-of-system seems to be mostly irrelevant, considering the delays of space travel and the fast time-scales a strong AI seems likely to operate at. (That is, assuming no FTL and the like.)

Do you have any plausible scenario in mind where an AI would be powerful enough to colonize the universe, but do it because it needs energy for doing something inside its system of origin?

I might see one perhaps extending to a few neighboring systems in a very dense cluster for some strange reason, but I can’t imagine likely final goals (again, for its birth star-system) that it would need to spend hundreds of millenia even to take over a single galaxy, let alone leave it. (Which is of course no proof there isn’t; my question above wasn’t rhethorical.)

I can imagine unlikely accidents causing some sort of papercliper-scenario, and maybe vanishingly rare cases where two or more AIs manage to fight each other over long periods of time, but it’s not obvious to me why this class of scenarios should be assigned a lot of probability mass in aggregate.

bogdanb 5 Sep 2014 23:06 UTC
0 points
in reply to: ChristianKl’s comment on: Memory is Everything
Honestly, I can’t really find anything significant in this comment I disagree with.

bogdanb 31 Aug 2014 13:41 UTC
0 points
in reply to: ChristianKl’s comment on: Memory is Everything

It’s a bit like opening a thread arguing that the Spanish inquisition was right for torturing nonbelievers because they they acted under the assumption that they could save souls from eternal damnation by doing so.

But the OP didn’t argue in support of torturing people, as far as I can tell. In the terms of your analogy, my reading was of the OP was a bit like:

“Hey, if the Spanish Inquisition came to you and offered the following two options, would you pick either of them, or refuse both? The options are (1) you’re excommunicated, then you get all the cake you want for a week, then you forget about it, or (2) you’re sanctified, then you’re tortured for a week, then you forget about it. Option (3) means nothing happens, they just leave.”

Which sounds completely different to my ears.

bogdanb 31 Aug 2014 8:56 UTC
0 points
in reply to: ChristianKl’s comment on: Memory is Everything
Sure, but then why do you expect memory and experience would also behave in a common sense manner? (At least, that’s what I think you did in your first comment.)

I interpreted the OP as “I’m confused about memory and experience; let’s try a thought experiment about a very uncommon situation just to see what we think it would happen”. And your first comment reads to me as “you picked a bad thought experiment, because you’re not describing a common situation”. Which seems to completely miss the point, the whole purpose of the thought experiment was to investigate the consequences of something very distinct from situations where “common sense” has real experience to rely on.

The part about torturing children I don’t even get at all. Wondering about something seems to me almost the opposite of the philosophy of “doing something because you think you know the answer”. Should we never do thought experiments, because someone might act on mistaken assumptions about those ideas? Not thinking about something before doing it sounds to me like exactly the opposite of the correct strategy.

bogdanb 31 Aug 2014 7:57 UTC
2 points
on: The Great Filter is early, or AI is hard

Once AI is developed, it could “easily” colonise the universe.

I was wondering about that. I agree with the could, but is there a discussion of how likely it is that it would decide to do that?

Let’s take it as a given that successful development of FAI will eventually lead to lots of colonization. But what about non-FAI? It seems like the most “common” cases of UFAI are mistakes in trying to create an FAI. (In a species with similar psychology to ours, a contender might also be mistakes trying to create military AI, and intentional creation by “destroy the world” extremists or something.)

But if someone is trying to create an FAI, and there is an accident with early prototypes, it seems likely that most of those prototypes would be programmed with only planet-local goals. Similarly, it doesn’t seem likely that intentionally-created weapon-AI would be programmed to care about what happens outside the solar system, unless it’s created by a civilization that already does, or is at least attempting, interstellar travel. Creators that care about safety will probably try to limit the focus, even imperfectly, both to make reasoning easier and to limit damage, and weapons-manufacturers will try to limit the focus for efficiency.

Now, I realize that a badly done AI could decide to colonize the universe even if its creators didn’t program it for that initially, and that simple goals can have that as an unforeseen consequence (like the prototypical paperclip manufacturer). But have we any discussion of how likely that is in a realistic setting? Perhaps the filter is that the vast majority of AIs limit themselves to their original solar system.

bogdanb 31 Aug 2014 7:38 UTC
3 points
in reply to: Peter Wildeford’s comment on: The Great Filter is early, or AI is hard
The problem with that is that life on Earth appeared about 4 billion years ago, while the Milky Way is more than 13 billion years old. If life were somewhat common, we wouldn’t expect to be the first, because there was time for it to evolve several times in succession, and it had lots of solar systems where it could have done it.

A possible answer could be that there was a very strong early filter during the first part of the Milky Way’s existence, and that filter lessened in intensity in the last few billion years.

The only examples I can think of are elemental abundance (perhaps in a young galaxy there are much fewer systems with diverse enough chemical compositions) and supernova frequency (perhaps a young galaxy is sterilized by frequent and large supernovas much more often than an older one’s). But AFAIK both of those variations can be calculated well enough for a Fermi estimate from what we know, so I’d expect someone who knows the subject much better than I would have made that point already if they were plausible answers.

bogdanb 31 Aug 2014 7:15 UTC
0 points
in reply to: ChristianKl’s comment on: Memory is Everything

Your rephrasing essentially says that you torture an identical copy of a person for a week.

If you read it carefully, my first rephrasing actually says that you torture the original person for a week, and then you (almost) perfectly erase their memories (and physical changes) during that week.

This is not changing the nature of the thought experiment in the OP; it is exactly the same experiment, plus a hypothetical example of how it could be achieved technically, because you implied that the experiment in the OP is impossible to achieve and thus ill-posed.

Or, at least, that’s how I interpreted “Of course I’m fighting the hypothetical thought experiment. I think the notion of experience without being affected doesn’t make any sense.” I just gave an example of how one can experience something and not be affected. It was a somewhat extreme example, but it seems appropriate when Omega is involved.

bogdanb 25 Aug 2014 20:29 UTC
0 points
in reply to: ChristianKl’s comment on: Memory is Everything
It seems rather silly to argue about that, when the thought experiment starts with Omega and bets for amounts of a billion dollars. That allows glossing over a lot of details. Your position is like objecting to a physics thought experiment that assumes frictionless surfaces, while the same thought experiment also assumes mass-less objects.

As a simple example: Omega might make a ridiculously precise scan of your entire body, subject you to the experiment (depending on which branch you chose), then restore each molecule to the same position and state it was during the initial scan, within the precision limits of the initial scan. Sure, there’ll be quantum uncertainty and such, but there’s no obvious reason why the differences would be greater than, say, the differences appearing during nodding off for a couple minutes. Omega even has the option of anesthetizing and freezing you during the scan and restoration, to reduce errors. You’d remember that part of the procedure, but you still wouldn’t be affected by what happened in-between.

(If you think about it, that’s very nearly equivalent to applying the conditions of the bet, with extremely high time acceleration, or while you’re suspended, to a very accurate simulation of yourself. The end effect is the same: an instance of you experienced torture/ultra-pampering for a week, and then an instance of you, which doesn’t remember the first part, experiences gaining/loosing a billion dollars.)

bogdanb 28 Jan 2014 19:30 UTC
6 points
in reply to: So8res’s comment on: Dark Arts of Rationality

perhaps costly, but worth the price

How about extending the metaphor and calling these techniques “Rituals” (they require a sacrifice, and even though it’s not as “permanent” as in HPMOR, it’s usually dangerous), reserving “Dark” for the arguably-immoral stuff?