Both trust handoff and decision handoff feel a bit slippery and “unfalisifiable” as concepts. What does it mean to automate the economy in a way that isn’t one or both? You say that ideally you’d want to do “trust handoff never thanks to some fancy scheme for AIs to watchdog each other”. But then your description of the “other extreme” of trust handoff sounds a lot like just such a fancy scheme. So is that trust handoff or not? I get that it’s a spectrum, but it seems really unhelpful to collapse this into a binary, and by extension into “an event” that happens at some point in time.
I feel like this slipperiness creates a lot of potential for miscommunication and semi-intentional motte-and-baileys here? E.g. if I say that we should just never do trust handoff, and someone else thinks it’s inevitable, it’s hard to tell if we’re even disagreeing on the object level? Relatedly, I’ve often heard people treat “handoff” as the inevitable end point of the development of superintelligence, and your definition seems to allow a kind of motte and bailey here where you can retreat to some unfalsifiable motte of “well if you automate the economy surely the AIs could take over if they all suddenly coordinated”, and then return to a bailey of “we’re inevitably going to do a really risky form of handoff”, without necessarily even noticing that that’s what you did.
Relatedly, if I try to think about which actions in human society constitute “trust handoff”, this feels very slippery and confusing? Are humans currently doing a lot of trust handoff to other humans? In some sense, sort of: If all of a company’s employees decided to just destroy the company, they could. The board and shareholders spend relatively little time overseeing most companies. But the overall societal incentives are very much against destroying the company, so I don’t think this is well described as “trust handoff”?
Extending the analogy to the level of an entire society: If there’s some class of shareholders, it’s probably the case that if all employees of the companies and the government coordinated against you, they could just expropriate you. And in that sense “the working class” collectively is in a position to take over the world if they all suddenly gained class consciousness and coordinated against the shareholder class. This is certainly a somewhat risky situation for the shareholder class to be in, but it doesn’t feel very natural to me to describe this as the shareholder class “handing off trust” to the working class? Indeed, for some definition of “shareholder class” and “working class”, the scenario above arguably already approximately describes our current world.
In general I worry that the phrase is importing a lot of implications of the word “trust” that are misleading?
Relatedly, it also seems kind of unlikely that anyone will ever make a decision that feels clearly like “I’m handing off trust now”? Maybe you disagree?
Thanks for the feedback! This is very helpful since we are deciding whether to use this terminology/framing going forward. I like your analogy to human society / the working class / etc. and am therefore somewhat dissatisfied with our current proposal and am curious to hear alternatives.
Can you say more about the motte and bailey you are worried people might do?
Do you have an alternative suggestion for what kinds of terminology we could use instead, to articulate the sorts of ideas we are trying to articulate?
I don’t think they are unfalsifiable as concepts fwiw, but I agree it would be nice if they were easier to measure.
On alternative terminology: It feels more natural to me to think about it in terms of the inverse, of how much oversight and how carefully-designed controls / checks and balances you have. Also importantly, how much centralization into “one” AI, vs. diverse competing AIs. As you have more powerful AI, to some extent it becomes by default more difficult to oversee, and in a competitive situation there’s a temptation to skimp on those checks and balances. This feels closer to how we think about it in the human case, e.g. there’s a temptation to remove checks and balances from the executive in exchange for getting the trains to run on time, a board might feel like they don’t want to interfere in their genius CEOs actions, etc.
And yeah I was using “unfalsifiable” in a very loose sense, mostly to point at the fact that, without a definition of where you draw the line between e.g. trust handoff and not-trust-handoff, it becomes impossible to definitively say that e.g. a prediction that we’d do “trust handoff” at a given point in time, was wrong.
Some kind of meta grumbling about this concept:
Both trust handoff and decision handoff feel a bit slippery and “unfalisifiable” as concepts. What does it mean to automate the economy in a way that isn’t one or both? You say that ideally you’d want to do “trust handoff never thanks to some fancy scheme for AIs to watchdog each other”. But then your description of the “other extreme” of trust handoff sounds a lot like just such a fancy scheme. So is that trust handoff or not? I get that it’s a spectrum, but it seems really unhelpful to collapse this into a binary, and by extension into “an event” that happens at some point in time.
I feel like this slipperiness creates a lot of potential for miscommunication and semi-intentional motte-and-baileys here? E.g. if I say that we should just never do trust handoff, and someone else thinks it’s inevitable, it’s hard to tell if we’re even disagreeing on the object level? Relatedly, I’ve often heard people treat “handoff” as the inevitable end point of the development of superintelligence, and your definition seems to allow a kind of motte and bailey here where you can retreat to some unfalsifiable motte of “well if you automate the economy surely the AIs could take over if they all suddenly coordinated”, and then return to a bailey of “we’re inevitably going to do a really risky form of handoff”, without necessarily even noticing that that’s what you did.
Relatedly, if I try to think about which actions in human society constitute “trust handoff”, this feels very slippery and confusing? Are humans currently doing a lot of trust handoff to other humans? In some sense, sort of: If all of a company’s employees decided to just destroy the company, they could. The board and shareholders spend relatively little time overseeing most companies. But the overall societal incentives are very much against destroying the company, so I don’t think this is well described as “trust handoff”?
Extending the analogy to the level of an entire society: If there’s some class of shareholders, it’s probably the case that if all employees of the companies and the government coordinated against you, they could just expropriate you. And in that sense “the working class” collectively is in a position to take over the world if they all suddenly gained class consciousness and coordinated against the shareholder class. This is certainly a somewhat risky situation for the shareholder class to be in, but it doesn’t feel very natural to me to describe this as the shareholder class “handing off trust” to the working class? Indeed, for some definition of “shareholder class” and “working class”, the scenario above arguably already approximately describes our current world.
In general I worry that the phrase is importing a lot of implications of the word “trust” that are misleading?
Relatedly, it also seems kind of unlikely that anyone will ever make a decision that feels clearly like “I’m handing off trust now”? Maybe you disagree?
Thanks for the feedback! This is very helpful since we are deciding whether to use this terminology/framing going forward. I like your analogy to human society / the working class / etc. and am therefore somewhat dissatisfied with our current proposal and am curious to hear alternatives.
Can you say more about the motte and bailey you are worried people might do?
Do you have an alternative suggestion for what kinds of terminology we could use instead, to articulate the sorts of ideas we are trying to articulate?
I don’t think they are unfalsifiable as concepts fwiw, but I agree it would be nice if they were easier to measure.
On alternative terminology: It feels more natural to me to think about it in terms of the inverse, of how much oversight and how carefully-designed controls / checks and balances you have. Also importantly, how much centralization into “one” AI, vs. diverse competing AIs. As you have more powerful AI, to some extent it becomes by default more difficult to oversee, and in a competitive situation there’s a temptation to skimp on those checks and balances. This feels closer to how we think about it in the human case, e.g. there’s a temptation to remove checks and balances from the executive in exchange for getting the trains to run on time, a board might feel like they don’t want to interfere in their genius CEOs actions, etc.
And yeah I was using “unfalsifiable” in a very loose sense, mostly to point at the fact that, without a definition of where you draw the line between e.g. trust handoff and not-trust-handoff, it becomes impossible to definitively say that e.g. a prediction that we’d do “trust handoff” at a given point in time, was wrong.