I was most confused about ‘S’, and likely understood it quite differently than intended.
I understood S as roughly “Humanity stays in control after AGI, but slowly (over decades/centuries) becomes fewer and less relevant”. I’d expect in many of these cases for something morally valuable to replace humans. So I put S lower than R.
Could it make sense to introduce “population after AGI you care about” as a term – I think this could be clearer.
I was most confused about ‘S’, and likely understood it quite differently than intended.
I understood S as roughly “Humanity stays in control after AGI, but slowly (over decades/centuries) becomes fewer and less relevant”. I’d expect in many of these cases for something morally valuable to replace humans. So I put S lower than R.
Could it make sense to introduce “population after AGI you care about” as a term – I think this could be clearer.