I followed the first part of this, and I agree: if V=min(S1) and U=min(S2) and S2 = S1 + a, and a is not an instrumental value on the way to something else in S1, then V-maximization will cause a (and therefore U) to approach zero.
You lost me when you introduced W.
Your conclusion seems to be saying that a system that optimizes for certain things will reliably optimize for those things, the hard part is building a system that optimizes for the things we want. I agree with that much, certainly.
I followed the first part of this, and I agree: if V=min(S1) and U=min(S2) and S2 = S1 + a, and a is not an instrumental value on the way to something else in S1, then V-maximization will cause a (and therefore U) to approach zero.
You lost me when you introduced W.
Your conclusion seems to be saying that a system that optimizes for certain things will reliably optimize for those things, the hard part is building a system that optimizes for the things we want. I agree with that much, certainly.
W is just a small error term on the utility function. A small error on S2 has a lot of consequences, an error on U has little.