Please read this paper before you start down an unproductive path. It’s in a different field but many of the same caveats apply.
The solution I came up with (when you insisted) was to verify rationality by observing the production of novel results. They don’t have to be Nobel-prize grade, but someone who has actually understood some material presented here should be able to use it to generate thoughts that are not the result of a cache lookup.
In practical terms, this would look like a “meta-exercise” a la “The Five Second Level”, asking the reader to first design an application exercise for the ideas newly assimilated, and then solve that exercise. The only requirement is that any solution should come as a surprise to the person proposing it.
verify rationality by observing the production of novel results. They don’t have to be Nobel-prize grade
I would expect even better results when monitoring catastrophes among the trained and the untrained. Things like “not going bankrupt” relative to those in a good control group, rather than “got tenure.”
Please read this paper before you start down an unproductive path. It’s in a different field but many of the same caveats apply.
The solution I came up with (when you insisted) was to verify rationality by observing the production of novel results. They don’t have to be Nobel-prize grade, but someone who has actually understood some material presented here should be able to use it to generate thoughts that are not the result of a cache lookup.
In practical terms, this would look like a “meta-exercise” a la “The Five Second Level”, asking the reader to first design an application exercise for the ideas newly assimilated, and then solve that exercise. The only requirement is that any solution should come as a surprise to the person proposing it.
I would expect even better results when monitoring catastrophes among the trained and the untrained. Things like “not going bankrupt” relative to those in a good control group, rather than “got tenure.”