Lets suppose that at this stage, GPT-17 has nanotech.
There are many things that you can suppose. You can also assume that GP-17 has no nanotech. Creating nanotech might require the development of highly complex machinery under particular conditions, and it might very well be that those nanotech factories are not ready by the time that this AGI is created.
Also, if you include low probability events where humans save the superintelligence (very low probability), then you should include the similarly unlikely scenarios where humans somehow harm the superintelligence
The way I see this is that the OP just presented a small piece of fiction. Why is he/she not allowed to imagine an unlikely scenario? It is even stated in the opening paragraph! Why does he/she have to present an equally unlikely scenario of the opposite sign?
I feel this reply is basically saying: I don’t like the text because it is not what I think is going to happen. But it is not really engaging with the scenario here presented, nor giving any real reasons or why the story presented here can’t happen this way.
One more thing: this shows to me how biased towards doomerism LW is. I think that If the OP had published a similar story but showing a catastrophic scenario, it probably would have received a much better reception.
There are many things that you can suppose. You can also assume that GP-17 has no nanotech. Creating nanotech might require the development of highly complex machinery under particular conditions, and it might very well be that those nanotech factories are not ready by the time that this AGI is created.
The way I see this is that the OP just presented a small piece of fiction. Why is he/she not allowed to imagine an unlikely scenario? It is even stated in the opening paragraph! Why does he/she have to present an equally unlikely scenario of the opposite sign?
I feel this reply is basically saying: I don’t like the text because it is not what I think is going to happen. But it is not really engaging with the scenario here presented, nor giving any real reasons or why the story presented here can’t happen this way.
One more thing: this shows to me how biased towards doomerism LW is. I think that If the OP had published a similar story but showing a catastrophic scenario, it probably would have received a much better reception.