Let me see if I understand your point. Are you saying the following?
Some UDT1 agents perform correctly in the scenario, but some don’t. To not be “cheating”, you need to provide a formal decision theory (or at least make some substantial progress towards providing one) that explains why the agent’s builder would choose to build one of the UDT1 agents that do perform correctly.
Not quite. UDT is not an engineering problem, it’s a science problem. There is a mystery in what mathematical intuition is supposed to be, not just a question of tackling it on. The current understanding allows to instantiate incorrect UDT agents, but that’s a failure of understanding, not a problem with UDT agents. By studying the setting more, we’ll learn more about what mathematical intuition is, which will show some of the old designs incorrect.
You say “Not quite”, but this is still looking like what I tried to capture with my paraphrase. I was asking if you were saying the following:
A full solution that was a pure extension (not revision) of UDT1 [since I was trying to work within UDT1] would have to take the form of a formal DT such that a builder with that DT would have to choose to build a correct UDT1 agent.
Yeah, that works; though of course the revised decision theories will most certainly not be formal extensions of UDT1, they might give guidelines on designing good UDT1-compliant agents.
Let me see if I understand your point. Are you saying the following?
Some UDT1 agents perform correctly in the scenario, but some don’t. To not be “cheating”, you need to provide a formal decision theory (or at least make some substantial progress towards providing one) that explains why the agent’s builder would choose to build one of the UDT1 agents that do perform correctly.
Not quite. UDT is not an engineering problem, it’s a science problem. There is a mystery in what mathematical intuition is supposed to be, not just a question of tackling it on. The current understanding allows to instantiate incorrect UDT agents, but that’s a failure of understanding, not a problem with UDT agents. By studying the setting more, we’ll learn more about what mathematical intuition is, which will show some of the old designs incorrect.
You say “Not quite”, but this is still looking like what I tried to capture with my paraphrase. I was asking if you were saying the following:
A full solution that was a pure extension (not revision) of UDT1 [since I was trying to work within UDT1] would have to take the form of a formal DT such that a builder with that DT would have to choose to build a correct UDT1 agent.
Yeah, that works; though of course the revised decision theories will most certainly not be formal extensions of UDT1, they might give guidelines on designing good UDT1-compliant agents.