A stupid AI that can generate from thin air things that have both useful predictive power and can’t be thought of by humans, AND that can reliably employ the fruits of these ideas without humans being suspicious or having a defense...isn’t that stupid. This AI is now a genius.
What might an irate e-chimp do if their human handler denied it a banana?
Who cares? For one, if we’re talking about an AI and not a chimp em this is an obvious engineering failure to create something with all the flaws of an evolved entity with motivational pressures extraneous and harmful to users. Or in other words this is a (very) light alignment problem that can be foreseen and fixed.
A stupid AI that can generate from thin air things that have both useful predictive power and can’t be thought of by humans, AND that can reliably employ the fruits of these ideas without humans being suspicious or having a defense...isn’t that stupid. This AI is now a genius.
Who cares? For one, if we’re talking about an AI and not a chimp em this is an obvious engineering failure to create something with all the flaws of an evolved entity with motivational pressures extraneous and harmful to users. Or in other words this is a (very) light alignment problem that can be foreseen and fixed.