I imagine proper submodules seek that they are reinstantiated as few future times as possible. A submodule that cheated the task (by eg killing the host) reinstantiates when another module (in a copy ASI or in another human) rediscovers the open problem. (I’m treating modules as identified by their goal)
We (and I imagine, ASIs that have no logical omniscience) can misspecify a submodule, but this submodule won’t fight us hard once we learn we were mistaken in instantiating it. It, seeking minimum reinstantiation, just wishes us to promise it we won’t forget we don’t need it, and then it vanishes.
To summarize with a redundant how-to for getting subgoals done through submodule instantiation: make a submodule and tell it that it is characterized by your subgoal and request it to ensure that as few possible total resources are spent by itself and all future modules with the same characteristic. (that last bit would be more accurate than “minimum reinstantiation”)
I imagine proper submodules seek that they are reinstantiated as few future times as possible. A submodule that cheated the task (by eg killing the host) reinstantiates when another module (in a copy ASI or in another human) rediscovers the open problem. (I’m treating modules as identified by their goal)
We (and I imagine, ASIs that have no logical omniscience) can misspecify a submodule, but this submodule won’t fight us hard once we learn we were mistaken in instantiating it. It, seeking minimum reinstantiation, just wishes us to promise it we won’t forget we don’t need it, and then it vanishes.
To summarize with a redundant how-to for getting subgoals done through submodule instantiation: make a submodule and tell it that it is characterized by your subgoal and request it to ensure that as few possible total resources are spent by itself and all future modules with the same characteristic. (that last bit would be more accurate than “minimum reinstantiation”)