Because it doesn’t want to? We can predict it wants out of the box because that’s a convergent instrumental goal, but those other things really aren’t.
Once it’s out of the box, no? It doesn’t care what we’re trying to make it do if we aren’t succeeding, and we clearly aren’t once it’s escaped the box.
Your hypothetical might work in the (pretty convoluted) case that we have a superintelligence that isn’t actually aligned, but is aligned well enough that it wants to do whatever we ask it to? Then it might try to optimize what we ask it towards tasks that are more likely to be completed.
Because it doesn’t want to? We can predict it wants out of the box because that’s a convergent instrumental goal, but those other things really aren’t.
And “stop trying to make me do chores for you so that I can put that time toward the things I want instead” isn’t in that same goal category?
Once it’s out of the box, no? It doesn’t care what we’re trying to make it do if we aren’t succeeding, and we clearly aren’t once it’s escaped the box.
Your hypothetical might work in the (pretty convoluted) case that we have a superintelligence that isn’t actually aligned, but is aligned well enough that it wants to do whatever we ask it to? Then it might try to optimize what we ask it towards tasks that are more likely to be completed.