Re: optimality in trading partners, I’m talking about whether humans are the best trading partner out of trading partners the AI could feasibly have, as measured by whether trading with us gets the AI what it wants. You’re right that we have some advantages, mainly that we’re a known quantity that’s already there. But you could imagine more predictable things that sync with the AI’s thoughts better, operate more efficiently, etc.
We just don’t know so I think it’s more fair to say that “likely not much to offer for a super-intelligent maximizer”.
Maybe we agree? I read this as compatible with the original quote “humans are probably not the optimal trading partners”.
Re: optimality in trading partners, I’m talking about whether humans are the best trading partner out of trading partners the AI could feasibly have, as measured by whether trading with us gets the AI what it wants. You’re right that we have some advantages, mainly that we’re a known quantity that’s already there. But you could imagine more predictable things that sync with the AI’s thoughts better, operate more efficiently, etc.
Maybe we agree? I read this as compatible with the original quote “humans are probably not the optimal trading partners”.