Sure. But same as DanielV’s point about elimiating circularity by just asking for the complete preference ordering, we are limited by what humans can think about.
Humans have to think in terms of high-level descriptions of approximately constant size, no matter the spatiotemporal scale. We literally cannot elicit preferences over universe-histories, much as we’d like to.
What we can do, maybe, is elicit some opinions on these “high-level descriptions of approximately constant size,” at many different spatiotemporal scales: ranging from general opinions on how the universe should go to what could improve the decor of your room today. Stitching these together into a utility function over universe histories is pretty tricky, but I think there might be some illuminating simplifications we could think about.
Sure. But same as DanielV’s point about elimiating circularity by just asking for the complete preference ordering, we are limited by what humans can think about.
Humans have to think in terms of high-level descriptions of approximately constant size, no matter the spatiotemporal scale. We literally cannot elicit preferences over universe-histories, much as we’d like to.
What we can do, maybe, is elicit some opinions on these “high-level descriptions of approximately constant size,” at many different spatiotemporal scales: ranging from general opinions on how the universe should go to what could improve the decor of your room today. Stitching these together into a utility function over universe histories is pretty tricky, but I think there might be some illuminating simplifications we could think about.