Cullen
Polio Lab Leak Caught with Wastewater Sampling
Thoughts on “The Offense-Defense Balance Rarely Changes”
The opening statements made it clear that no one involved cared about or was likely even aware of existential risks.
I think this is a significant overstatement given, especially, these remarks from Sen. Hawley:
And I think my question is, what kind of an innovation is [AI] going to be? Is it gonna be like the printing press that diffused knowledge, power, and learning widely across the landscape that empowered, ordinary, everyday individuals that led to greater flourishing, that led above all two greater liberty? Or is it gonna be more like the atom bomb, huge technological breakthrough, but the consequences severe, terrible, continue to haunt us to this day? I don’t know the answer to that question. I don’t think any of us in the room know the answer to that question. Cause I think the answer has not yet been written. And to a certain extent, it’s up to us here and to us as the American people to write the answer.
Obviously he didn’t use the term “existential risk.” But that’s not the standard we should use to determine whether people are aware of risks that could be called, in our lingo, existential. Hawley clearly believes that there is a clear possibility that this could be an atomic-bomb-level invention, which is pretty good (but not decisive) evidence that, if asked, he would agree that this could cause something like human extinction.
Law-Following AI 1: Sequence Introduction and Structure
Tracking Compute Stocks and Flows: Case Studies?
AI Benefits Post 1: Introducing “AI Benefits”
Parallels Between AI Safety by Debate and Evidence Law
Antitrust-Compliant AI Industry Self-Regulation
AI Benefits Post 3: Direct and Indirect Approaches to AI Benefits
AI Benefits Post 2: How AI Benefits Differs from AI Alignment & AI for Good
Law-Following AI 4: Don’t Rely on Vicarious Liability
Law-Following AI 2: Intent Alignment + Superintelligence → Lawless AI (By Default)
AI Benefits Post 4: Outstanding Questions on Selecting Benefits
AI Benefits Post 5: Outstanding Questions on Governing Benefits
Good point!
Thanks!
The offensive technology to make the planet unsuitable for current human civilization ALREADY exists—the defense so far has consisted of convincing people not to use it.
I think this is true in the limit (assuming you’re referring primarily to nukes). But I think offense-defense reasoning is still very relevant here: For example, to know when/how much to worry about AIs using nuclear technology to cause human extinction, you would want to ask under what circumstances can humans defend command and control of nuclear weapons from AIs that want to seize them.
We just can’t learn much from human-human conflict, where at almost any scale, the victor hopes to have a hospitable environment remaining afterward.
I agree that the calculus changes dramatically if you assume that the AI does not need or want the earth to remain inhabitable by humans. I also agree that, in the limit, interspecies interactions are plausibly a better model than human-human conflicts. But I don’t agree that either of these implies that offense-defense logic is totally irrelevant.
Humans, as incumbents, inherently occupy the position of defenders as against the misaligned AIs in these scenarios, at least if we’re aware of the conflict (which I grant we might not be). The worry is that AIs will try to gain control in certain ways. Offense-defense thinking is important if we ask questions like:
Can we predict how AIs might try to seize control? I.e., what does control consist in from their perspective, and how might they achieve that given parties’ starting positions.
If we have some model of how AIs try to seize control, what does that imply about humanity’s ability to defend itself?
Thanks, done. LW makes it harder than EAF to make sequences, so I didn’t realize any community member could do so.
-
I haven’t read all of Asimov, but in general, “the” law has a much richer body of interpretation and application than the Laws of Robotics did, and also have authoritative, external dispute resolution processes.
-
I don’t think so. The Counselor function is just a shorthand for the process of figuring out how the law might fairly apply to X. An agent may or may not have the drive to figure that out by default, but the goal of an LFAI system is to give it that motivation. Whether it figures out the law by asking another agent or simply reasoning about the law itself is ultimately not that important.
-
I had one of the EA Forum’s launch codes, but I decided to permanently delete it as an arms-reduction measure. I no longer have access to my launch code, though I admit that I cannot convincingly demonstrate this.