It might be fun to pair Humankind: A Hopeful History with The Precipice, as both have been suggested reading recently.
It seems to me that we are, as individuals, getting more and more powerful. So this question of “alignment” is a quite important one— as much for humanity, with the power it currently has, as for these hypothetical hyper-intelligent AIs.
Looking at it through a Sci-Fi AI lens seems limiting, and I still haven’t really found anything more than “the future could go very very badly”, which is always a given, I think.
I’ve read those papers you linked (thanks!). They seem to make some assumptions about the nature of intelligence, and rationality— indeed, the nature of reality itself. (Perhaps the “reality” angle is a bit much for most heads, but the more we learn, the more we learn we need to learn, as it were. Or at least it seems thus to me. What is “real”? But I digress) I like the idea of Berserkers (Saberhagen) better than run amok Pi calculators… however, I can dig it. Self-replicating killer robots are scary. (Just finished Horizon: Zero Dawn—Forbidden West and I must say it was as fantastic as the previous installment!)
Which of the AI books would you recommend I read if I’m interested in solutions? I’ve read a lot of stuff on this site about AI now (before I’d read mostly Sci-Fi or philosophy here, and I never had an account or interacted), most of it seems to be conceptual and basically rephrasing ideas I’ve been exposed to through existing works. (Maybe I should note that I’m a fan of Kurzweil’s takes on these matters— takes which don’t seem to be very popular as of late, if they ever were. For various reasons, I reckon. Fear sells.) I assume Precipice has some uplifting stuff at the end[1], but I’m interested in AI specifically ATM.
What I mean is, I’ve seen a few of proposals to “ensure” alignment, if you will, with what we have now (versus say warnings to keep in mind once we have AGI or are demonstrably close to it). One is that we start monitoring all compute resources. Another is that we start registering all TPU (and maybe GPU) chips and what they are being used for. Both of these solutions seem scary as hell. Maybe worse than replicating life-eating mecha, since we’ve in essence experienced ideas akin to the former a few times historically. (Imagine if reading was the domain of a select few and books were regulated!)
If all we’re talking about with alignment here, really, is that folks need keep in mind how bad things can potentially go, and what we can do to be resilient to some of the threats (like hardening/distributing our power grids, hardening water supplies, hardening our internet infrastructure, etc.), I am gung-ho!
On the other hand, if we’re talking about the “solutions” I mentioned above, or building “good” AIs that we can use to be sure no one is building “bad” AIs, or requiring the embedding of “watermarks” (DRM) into various “AI” content, or building extending sophisticated communication monitoring apparatus, or other such — to my mind — extremely dangerous ideas, I’m thinking I need to maybe convince people to fight that?
In closing, regardless of what the threats are, be they solar flares or comets (please don’t jinx us!) or engineered pathogens (intentional or accidental) or rogue AIs yet to be invented — if not conceived of —, a clear “must be done ASAP” goal is colonization of places besides the Earth. That’s part of why I’m so stoked about the future right now. We really seem to be making progress after stalling out for a grip. Guess the same goes for AI, but so far all I see is good stuff coming from that forward motion too. A little fear is good! but too much? not so much.
I really like the idea of 80,000 Hours, and seeing it mentioned in the FAQ for the book, so I’m sure there are some other not-too-shabby ideas there. I oft think I should do more for the world, but truth be told (if one cannot tell from my writing), I barely seem able to tend my own garden.
It might be fun to pair Humankind: A Hopeful History with The Precipice, as both have been suggested reading recently.
It seems to me that we are, as individuals, getting more and more powerful. So this question of “alignment” is a quite important one— as much for humanity, with the power it currently has, as for these hypothetical hyper-intelligent AIs.
Looking at it through a Sci-Fi AI lens seems limiting, and I still haven’t really found anything more than “the future could go very very badly”, which is always a given, I think.
I’ve read those papers you linked (thanks!). They seem to make some assumptions about the nature of intelligence, and rationality— indeed, the nature of reality itself. (Perhaps the “reality” angle is a bit much for most heads, but the more we learn, the more we learn we need to learn, as it were. Or at least it seems thus to me. What is “real”? But I digress) I like the idea of Berserkers (Saberhagen) better than run amok Pi calculators… however, I can dig it. Self-replicating killer robots are scary. (Just finished Horizon: Zero Dawn—Forbidden West and I must say it was as fantastic as the previous installment!)
Which of the AI books would you recommend I read if I’m interested in solutions? I’ve read a lot of stuff on this site about AI now (before I’d read mostly Sci-Fi or philosophy here, and I never had an account or interacted), most of it seems to be conceptual and basically rephrasing ideas I’ve been exposed to through existing works. (Maybe I should note that I’m a fan of Kurzweil’s takes on these matters— takes which don’t seem to be very popular as of late, if they ever were. For various reasons, I reckon. Fear sells.) I assume Precipice has some uplifting stuff at the end[1], but I’m interested in AI specifically ATM.
What I mean is, I’ve seen a few of proposals to “ensure” alignment, if you will, with what we have now (versus say warnings to keep in mind once we have AGI or are demonstrably close to it). One is that we start monitoring all compute resources. Another is that we start registering all TPU (and maybe GPU) chips and what they are being used for. Both of these solutions seem scary as hell. Maybe worse than replicating life-eating mecha, since we’ve in essence experienced ideas akin to the former a few times historically. (Imagine if reading was the domain of a select few and books were regulated!)
If all we’re talking about with alignment here, really, is that folks need keep in mind how bad things can potentially go, and what we can do to be resilient to some of the threats (like hardening/distributing our power grids, hardening water supplies, hardening our internet infrastructure, etc.), I am gung-ho!
On the other hand, if we’re talking about the “solutions” I mentioned above, or building “good” AIs that we can use to be sure no one is building “bad” AIs, or requiring the embedding of “watermarks” (DRM) into various “AI” content, or
buildingextending sophisticated communication monitoring apparatus, or other such — to my mind — extremely dangerous ideas, I’m thinking I need to maybe convince people to fight that?In closing, regardless of what the threats are, be they solar flares or comets (please don’t jinx us!) or engineered pathogens (intentional or accidental) or rogue AIs yet to be invented — if not conceived of —, a clear “must be done ASAP” goal is colonization of places besides the Earth. That’s part of why I’m so stoked about the future right now. We really seem to be making progress after stalling out for a grip.
Guess the same goes for AI, but so far all I see is good stuff coming from that forward motion too.
A little fear is good! but too much? not so much.
I really like the idea of 80,000 Hours, and seeing it mentioned in the FAQ for the book, so I’m sure there are some other not-too-shabby ideas there. I oft think I should do more for the world, but truth be told (if one cannot tell from my writing), I barely seem able to tend my own garden.