a system is ascription universal if, relative to our current epistemic state, its explicit beliefs contain just as much information as any other way of ascribing beliefs to it.
This is a bit different than the definition in my post (which requires epistemically dominating every other simple computation), but is probably a better approach in the long run. This definition is a little bit harder to use for the arguments in this post, and my current expectation is that the “right” definition will be usable for both informed oversight and securing HCH. Within OpenAI Geoffrey Irving has been calling a similar property “belief closure.”
This is not necessarily a difficult concern to address in the sense of making sure that any definition of ascription universality includes some concept of ascribing beliefs to a system by looking at the beliefs of any systems that helped create that system.
The memoized table is easy to epistemically dominate. To the extent that malicious cognition went into designing the table, you can ignore it when evaluating epistemic dominance.
The training process that produced the memoized table can be hard to epistemically dominate. That’s what we should be interested in. (The examples in the post have this flavor.)
This is a bit different than the definition in my post (which requires epistemically dominating every other simple computation), but is probably a better approach in the long run. This definition is a little bit harder to use for the arguments in this post, and my current expectation is that the “right” definition will be usable for both informed oversight and securing HCH. Within OpenAI Geoffrey Irving has been calling a similar property “belief closure.”
In the language of my post, I’d say:
The memoized table is easy to epistemically dominate. To the extent that malicious cognition went into designing the table, you can ignore it when evaluating epistemic dominance.
The training process that produced the memoized table can be hard to epistemically dominate. That’s what we should be interested in. (The examples in the post have this flavor.)