If the current trajectory continues, it’s not the case that the AI you have is a faithful representative of you, personally, run in your garage. Rather it seems there is a complex socio-economic process leading to the creation of the AIs, and the smarter they are, the more likely it is they were created by a powerful company or a government.
This process itself shapes what the AIs are “aligned” to. Even if we solve some parts of the technical alignment problem we still face the question of what is the sociotechnical process acting as “principal”.
This touches an idea I’d really like to get more attention. The idea that we should build AI fundamentally tethered to human nature so that this drift toward arbitrary form that e.g. “just so happens to sell best” doesn’t happen. I call it tetherware—more about in my post here.
This touches an idea I’d really like to get more attention. The idea that we should build AI fundamentally tethered to human nature so that this drift toward arbitrary form that e.g. “just so happens to sell best” doesn’t happen. I call it tetherware—more about in my post here.