It may be overly abstract. I’m a philosopher by training and I have a tendency to get overly abstract (which I am working on).
I agree that there are important possibilities with threshold effects, such as extinction and perhaps including your point about threshold effects with indirect normativity AIs. I also think that other scenarios, such as Robin Hanson’s scenario, other decentralized market/democracy set-ups, and other scenarios we can’t think of are live possibilities. More continuous trajectory changes may be very relevant in these other scenarios.
It may be overly abstract. I’m a philosopher by training and I have a tendency to get overly abstract (which I am working on).
I agree that there are important possibilities with threshold effects, such as extinction and perhaps including your point about threshold effects with indirect normativity AIs. I also think that other scenarios, such as Robin Hanson’s scenario, other decentralized market/democracy set-ups, and other scenarios we can’t think of are live possibilities. More continuous trajectory changes may be very relevant in these other scenarios.
For what it’s worth, I loved this post and don’t think it was very abstract. Then again, my background is also in philosophy.