Strong upvote, this is amazing to me. On the post:
Another example of explaining the intuitions for formal results less formally. I strongly support this as a norm.
I found the graphics helpful, both in style and content.
Some thoughts on the results:
This strikes at the heart of AI risk, and to my inexpert eyes the lack of anything rigorous to build on or criticize as a mechanism for the flashiest concerns has been a big factor in how difficult it was and is to get engagement from the rest of the AI field. Even if the formalism fails due to a critical flaw, the ability to spot such a flaw is a big step forward.
The formalism of average attainable utility, and the explicit distinction from number of possibilities, provides powerful intuition even outside the field. This includes areas like warfare and business. I realize it isn’t the goal, but I have always considered applicability outside the field as an important test because it would be deeply concerning for thinking about goal-directed behavior to mysteriously fail when applied to the only extant things which pursue goals.
I find the result aesthetically pleasing. This is not important, but I thought I would mention it.
Strong upvote, this is amazing to me. On the post:
Another example of explaining the intuitions for formal results less formally. I strongly support this as a norm.
I found the graphics helpful, both in style and content.
Some thoughts on the results:
This strikes at the heart of AI risk, and to my inexpert eyes the lack of anything rigorous to build on or criticize as a mechanism for the flashiest concerns has been a big factor in how difficult it was and is to get engagement from the rest of the AI field. Even if the formalism fails due to a critical flaw, the ability to spot such a flaw is a big step forward.
The formalism of average attainable utility, and the explicit distinction from number of possibilities, provides powerful intuition even outside the field. This includes areas like warfare and business. I realize it isn’t the goal, but I have always considered applicability outside the field as an important test because it would be deeply concerning for thinking about goal-directed behavior to mysteriously fail when applied to the only extant things which pursue goals.
I find the result aesthetically pleasing. This is not important, but I thought I would mention it.