i don’t really feel either imperils the centre of the argument
I think the center of the argument is basically correct
as for the “multiple agents, all hobbled by an unchanging terminal goal”: well, they’ll be outcompeted by the first one that gives it up.
This is not the scenario I’m imagining. I’m imagining multiple agents, some with thin terminal goals and others concerned purely with Omohundro drives, operating in context where the rational thing to do is the same whether or not you have non-Omohundro terminal goals.
In this case agents with thin terminal goals are not hobbled and they will not be out competed.
those are valid objections, but i don’t really feel either imperils the centre of the argument. i have touched upon the singleton side here.
as for the “multiple agents, all hobbled by an unchanging terminal goal”: well, they’ll be outcompeted by the first one that gives it up.
I think the center of the argument is basically correct
This is not the scenario I’m imagining. I’m imagining multiple agents, some with thin terminal goals and others concerned purely with Omohundro drives, operating in context where the rational thing to do is the same whether or not you have non-Omohundro terminal goals.
In this case agents with thin terminal goals are not hobbled and they will not be out competed.
I understand. I think in that case, the risk i argued against (nothing of value in the world) would still be avoided (at least within my ontology).