Note that the reprogenetics program is extremely achievable with ordinary biological science. There’s a relatively large pool of talented geneticists who are unlikely to have the security mindset necessary to directly work on capable AI, so I don’t think this program will detract human capital from other cause areas relevant to AI survivability. The risk does not extend far past the experimental subjects. It’s low cost with potentially huge upside. And it’s not like having healthier and smarter children is a thing people would be opposed to absent AI risk.
I also agree with the RobertM comment that the substantial impact with short timelines is well-argued. I think it’s worth adding that even if this program is doomed to fail because we get clotheslined by short timelines before it can do anything, it would be more dignified to die having started trying to become capable of solving alignment that it would be to be killed by misalignment without trying anything even conditioned on being correct that there’s predictably not enough time to complete the program. This is not as important as the argument that we should play to our outs, but it makes me feel better.
Note that the reprogenetics program is extremely achievable with ordinary biological science. There’s a relatively large pool of talented geneticists who are unlikely to have the security mindset necessary to directly work on capable AI, so I don’t think this program will detract human capital from other cause areas relevant to AI survivability. The risk does not extend far past the experimental subjects. It’s low cost with potentially huge upside. And it’s not like having healthier and smarter children is a thing people would be opposed to absent AI risk.
I also agree with the RobertM comment that the substantial impact with short timelines is well-argued. I think it’s worth adding that even if this program is doomed to fail because we get clotheslined by short timelines before it can do anything, it would be more dignified to die having started trying to become capable of solving alignment that it would be to be killed by misalignment without trying anything even conditioned on being correct that there’s predictably not enough time to complete the program. This is not as important as the argument that we should play to our outs, but it makes me feel better.