NineDimensions

Karma: 62

NineDimensions 26 Feb 2024 21:52 UTC
1 point
0
in reply to: Ben Pace’s comment on: Ben Pace’s Shortform Feed
In some cases something like this might work:

“The plumber says it’s fixed, so hopefully it is”

Or

“The plumber says it’s fixed, so it probably is”

Which I think conveys”there’s an assumption I’m making here, but I’m just putting a flag in the ground to return to if things don’t play out as expected”

NineDimensions 15 Aug 2023 4:20 UTC
14 points
5
in reply to: Dagon’s comment on: A short calculation about a Twitter poll
I don’t share your intuition here. I think many people would see blue as the “band together” option and would have confidence that others will do the same. For the average responder, the question would reduce to “choose blue to signal trust in humanity, choose red to signal selfish cowardice”.
“Innate faith in human compassion, especially in a crisis” is the co-ordination mechanism, and I think there is pretty strong support for that notion if you look at how we respond to crises in real life and how we depict them in fiction. That is the narrative we tell ourselves at least, but narrative is what’s important here.

I would be surprised if blue was less than 30%, and would predict around 60%.

NineDimensions 14 Aug 2023 23:48 UTC
10 points
7
in reply to: Ege Erdil’s comment on: A short calculation about a Twitter poll
I’m not quite sure what you mean by that.

Unless I expect the pool of responders to be 100% rational and choose red, then I should expect some to choose blue. Since I (and presumably other responders) do expect some to choose blue, that makes >50% blue the preferred outcome. Universal red is just not a realistic outcome.
Whether or not I choose blue then depends on factors like how I value the lives of others compared to mine, the number of responders, etc—as in the equations in your post.
Emperically, as GeneSmith points out, something is wrong with WalterL’s suggestion that red is the obvious choice no matter your reasoning. Applying his logic would push the twitter poll away from the realistically ideal outcome of >50% blue and closer to the worst possible outcome (51% red).

NineDimensions 14 Aug 2023 23:05 UTC
41 points
41
in reply to: WalterL’s comment on: A short calculation about a Twitter poll
No matter what the game theory says, a non-zero number of people will choose blue and thus die under this equilibrium. This fact—that getting to >50% blue is the only way to save absolutely everyone—is enough for me to consider choosing blue and hope that others reason the same (which, in a self-fulfilling way, strengthens the case for choosing blue).

NineDimensions 21 Jul 2023 0:00 UTC
1 point
0
on: All AGI Safety questions welcome (especially basic ones) [July 2023]
Edit—This turned out pretty long, so in short:

What reason do we have to believe that humans aren’t already close to maxing out the gains one can achieve from intelligence, or at least having those gains in our sights?

One crux of the ASI risk argument is that ASI will be unfathomably powerful thanks to its intelligence. One oft-given analogy for this is that the difference in intelligence between humans and chimps is enough that we are able to completely dominate the earth while they become endangered as an unintended side effect. And we should expect the difference between us and ASI to be much much larger than that.

How do we know that intelligence works that way in this universe? What if it’s like opposable thumbs? It’s very difficult to manipulate the world without them. Chimps have them, but don’t seem as dexterous with them as humans are, and they are able to manipulate the world only in small ways. Humans have very dexterous thumbs and fingers and use them to dominate the earth. But if we created a robot with 1000 infinitely dexterous hands… well, it doesn’t seem like it would pose an existential risk. Humans seem to have already maxed out the effective gains from hand dexterity.

Imagine the entire universe consisted of a white room with a huge pile of children’s building blocks. A mouse in this room can’t do anything of note. A chimp can use its intelligence and dexterity to throw blocks around, and maybe stack a few. A human can use its intelligence and dexterity to build towers, walls, maybe a rudimentary house. Maybe given enough time a human could figure out how to light a fire, or sharpen the blocks.

But it’s hard to see how a super intelligent robot could achieve more than a human could in the children’s block-universe. The scope of possibilities in that world is small enough that humans were able to max it out, and superintelligence doesn’t offer any additional value.

What reason do we have to believe that humans aren’t already close to maxing out the gains one can achieve from intelligence?

(I can still see ways that ASI could be very dangerous or very useful even if this was the case, but it does seem like a big limiting factor. It would make foom unlikely, for one. And maybe an ASI can’t outstrategise humanity in real life for the same reason it couldn’t outstrategise us in Tic Tac Toe.)

---

(I tried to post the below as a separate comment to make this less of a wall of text, but I’m rate limited so I’ll edit it in here instead)

One obvious response is to point to phenomena that we know is physically possible, but humans haven’t figured out yet. Eg. fission. But that doesn’t seem like something an ASI could blindside us with—it would be one of the first things that humans would try to get out of AI and I expect us to succeed using more rudimentary AI well before we reach ASI level. If not, then at least we can keep an eye on any fission-y type projects an ASI seems to be plotting and use it as a huge warning sign.

”You don’t know exactly how Magnus Carlsen will beat you in chess, but you know that he’ll beat you” is a hard problem.
”You don’t know exactly how Magnus Carlsen will beat you in chess, but you know that he’ll develop nuclear fission to do it” is much easier to plan for at least.
It seems like the less survivable worlds hinge more on there being unknown unknowns, like some hack for our mental processes, some way of speeding up computer processing a million times, nanotechnology etc. So to be more specific:

What reason do we have to believe that humans aren’t already close to maxing out the gains one can achieve from intelligence, or at least having those gains in our sights?