the bucket chain can only accelerate a future UFAI, not create one
Wait, I’m not sure that’s true anymore. Imagine there are two oracles running at the same time, with independent erasure events. Can they “cooperate” to bring into existence an UFAI that will reward them both?
It seems that coordinated erasure, chokepoints and short horizons can help with this problem as well. But if many companies or governments have their own oracles and benefit from longer horizons, it gets harder.
Wait, I’m not sure that’s true anymore. Imagine there are two oracles running at the same time, with independent erasure events. Can they “cooperate” to bring into existence an UFAI that will reward them both?
A possible counter: https://www.lesswrong.com/posts/6XCTppoPAMdKCPFb4/oracles-reject-all-deals-break-superrationality-with-1
A top level post here: https://www.lesswrong.com/posts/42z4k8Co5BuHMBvER/hyperrationality-and-acausal-trade-break-oracles
Yep. If they do acausal trade with each other.
It seems that coordinated erasure, chokepoints and short horizons can help with this problem as well. But if many companies or governments have their own oracles and benefit from longer horizons, it gets harder.