This actually generalises beyond mathematics quite well—rather than ask people to describe a theory let them apply the theory to an example (perhaps even an example that actually happened).
As I haven’t encountered this too often in practice (outside of mathematics, where proving for an exam is the rule rather than the exception) the obvious question is: why isn’t this being used?
the obvious question is: why isn’t this being used?
These are the moments when I am not sure… the obvious answer is “because people are mostly insane”… but then I feel guilty for thinking about this… but even after reflection this seems like the most likely explanation. (Okay, I could use more polite words and replace “insane” by “having really fucked up epistemic habits”. Just to emphasise that the problem isn’t in their hardware, it’s just the software that is completely crap.)
A random data point: When I studied psychology, when we learned about Freud’s ego-defense mechanisms, I was like: wow, this is finally something you could make a “math-style” exam about. (Okay, I know Freud is unpopular here, but please suspend your judgement for the sake of argument.) Because the ego-defense mechanisms have a relatively clear definition. E.g., someone says: “you ” and you reply: “no, you and !!!” (without there being any evidence for the other person ); this is called “projection”. Or, someone says: “you ”, and you reply: “no, actually I ” (without any evidence for , and actually some evidence for ), that’s called “reaction formation”. Etc.
So I was like: This is perfect. Here is how I would make an exam: Write a short story illustrating a use of an ego-defense mechanism, and then ask which of the mechanisms it was. That should be super easy, like elementary-school difficulty, and yet would help to connect the definition with a specific example (which some of my classmates had a problem to do; they just memorized the list of names of the ego-defense mechanisms, without attaching any specific meaning to those words; because memorizing lists was the generally used strategy there). I tried to explain this idea to some classmates, I tried to explain it to a teacher… no success. The typical response was something like: “Yes, such a thing could be done, but what’s the point? How is that an improvement over memorizing the list of names? It just takes more time to do.” (Except for one girl who studied both psychology and math, and she said: “Yes! Obviously, that’s how it should be done.”)
So, my conclusion is that some people are so epistemically challenged that they don’t even understand the difference between understanding something and not understanding but remembering the passwords. (Did I say “some people”? I meant a huge majority of people, including a huge majority of people with university education, including a huge majority of people who specifically study human thinking and/or education at the university!)
Things like this leave me sad about the mental state of the humankind. (And then I read an alternative explanation, that it’s all just a cultish influence of LW making me intolerant towards non-rationalists. No, it’s a fucking epistemic abyss. The only thing LW did to me was to point a finger to that abyss and say: “Yeah, we see it, too. You are not halucinating. It really is bad.”)
In mathematics, you can have people solve a problem, which is not just repeating the right password, but actually using the knowledge.
But this is probably exceptional for mathematics. :(
This actually generalises beyond mathematics quite well—rather than ask people to describe a theory let them apply the theory to an example (perhaps even an example that actually happened).
As I haven’t encountered this too often in practice (outside of mathematics, where proving for an exam is the rule rather than the exception) the obvious question is: why isn’t this being used?
These are the moments when I am not sure… the obvious answer is “because people are mostly insane”… but then I feel guilty for thinking about this… but even after reflection this seems like the most likely explanation. (Okay, I could use more polite words and replace “insane” by “having really fucked up epistemic habits”. Just to emphasise that the problem isn’t in their hardware, it’s just the software that is completely crap.)
A random data point: When I studied psychology, when we learned about Freud’s ego-defense mechanisms, I was like: wow, this is finally something you could make a “math-style” exam about. (Okay, I know Freud is unpopular here, but please suspend your judgement for the sake of argument.) Because the ego-defense mechanisms have a relatively clear definition. E.g., someone says: “you ” and you reply: “no, you and !!!” (without there being any evidence for the other person ); this is called “projection”. Or, someone says: “you ”, and you reply: “no, actually I ” (without any evidence for , and actually some evidence for ), that’s called “reaction formation”. Etc.
So I was like: This is perfect. Here is how I would make an exam: Write a short story illustrating a use of an ego-defense mechanism, and then ask which of the mechanisms it was. That should be super easy, like elementary-school difficulty, and yet would help to connect the definition with a specific example (which some of my classmates had a problem to do; they just memorized the list of names of the ego-defense mechanisms, without attaching any specific meaning to those words; because memorizing lists was the generally used strategy there). I tried to explain this idea to some classmates, I tried to explain it to a teacher… no success. The typical response was something like: “Yes, such a thing could be done, but what’s the point? How is that an improvement over memorizing the list of names? It just takes more time to do.” (Except for one girl who studied both psychology and math, and she said: “Yes! Obviously, that’s how it should be done.”)
So, my conclusion is that some people are so epistemically challenged that they don’t even understand the difference between understanding something and not understanding but remembering the passwords. (Did I say “some people”? I meant a huge majority of people, including a huge majority of people with university education, including a huge majority of people who specifically study human thinking and/or education at the university!)
Things like this leave me sad about the mental state of the humankind. (And then I read an alternative explanation, that it’s all just a cultish influence of LW making me intolerant towards non-rationalists. No, it’s a fucking epistemic abyss. The only thing LW did to me was to point a finger to that abyss and say: “Yeah, we see it, too. You are not halucinating. It really is bad.”)