(This comment is me trying to generate my own signs of successful traditions of knowledge, before reading most of the post. Is fairly rambly #BetterWrittenQuicklyThanNotWrittenAtAll)
My first thought on what signs to use when looking for the life of traditions, is to borrow from models of hiring (a company’s hiring process is what determines whether the core insights of a company have been successfully shared to all its employees, so it seemed analogous). Some quick examples:
Does it depend on short interviews or extended trial work with current core members of the org? If the former, the company culture will be eroded quickly, and if the latter, much less likely.
Does the hiring process heavily use the contacts of the current employees? Using top recommendations from existing teammates can quickly zoom in on those with the most appropriate competencies and thinking patterns.
For intellectual traditions, I’d look to how much selection the practitioners of the art had on who followed them, and whether they were able to train them a great deal. For example, if you’re trying to continue a tradition of building compilers a certain way at Stanford, if you’re a professor there you’ll get to choose your postdocs from a wide set of excellent PhD graduates. If you’re able to work with a number of them first too, then you get a really strong sense of how they all think. An extra fun thing is that you’re able (with such a technical field as CS) to pick technical problems that you give people when they apply, and that you swear them to secrecy on, so that nobody can learn them in advance and practice for them. Pick a question that’s a good indicator of the core technical understanding, and you’re off to the races.
Any process of passing on knowledge that doesn’t have these attributes will be correspondingly worse.
Another thought on this is from HPMOR. As it says in the book, the difference between Minister for Magic, and Order of Merlin is that successive Ministers are voted in by the wizarding public, whereas the Order of Merlin is handed on by personal choice of the previous holder of the Order of Merlin at their death—and this is why the Minister of Magic is Cornelius Fudge, and the Order of Merlin is Albus Dumbledore. An unbroken line from Merlin himself. This is related to the students of Noble Prize Winners often being incredible themselves (something I’ve been told but never verified myself FYI).
So these are hiring analogies. I expect that most of Samo’s examples will be natural to this framework. If not… then it’s probably due to looking at different scales of the process. For example, you can guess if a company has a good hiring process if they’re successful, so you can look back at great companies and suppose they’re successful. Analogously, if a field (e.g. mathematics) consistently produces great insights over the course of a century, you can guess that the tradition was alive. And at the other end of the scale, you can probably look at the general rationality signs of the person doing the hiring, as opposed to the process itself.
Okay, that’s my brainstorming.
goes and reads post
Thoughts:
I notice I had thought some of the points, and dismissed them. For example having shared methodology, concepts or conceptual framework seemed to me too easy to goodhart on, so I dismissed them.
However, (a) they are still bayesian evidence toward the tradition being passed on correctly, and (b) if there is little incentive for people to game the system, then they’re perfectly good metrics.
I passingly mentioned judging output, and I agree with the OP that this is in fact the strongest signal. (I used a similar one myself here.)
Existence of a physical location did not even slightly enter my mind. But it totally fits the company framing—I’ve worked on remote teams before and they’re terrible at producing quality work together and transferring insights to each other.
Overall I had an okay basic framework, but the post had a delightful amount of detail that I either hadn’t noticed or hadn’t made explicit, so I give myself a C+ grade. (And now I’m super excited for sequels to this post!)
Excellent exercise! It seems a good way to evaluate content, initially building out or explicating one’s own models before comparing it to someone else’s, I will remember to do this.
I’m happy to hear you’ve found the model I present on useful and interesting!
(This comment is me trying to generate my own signs of successful traditions of knowledge, before reading most of the post. Is fairly rambly #BetterWrittenQuicklyThanNotWrittenAtAll)
My first thought on what signs to use when looking for the life of traditions, is to borrow from models of hiring (a company’s hiring process is what determines whether the core insights of a company have been successfully shared to all its employees, so it seemed analogous). Some quick examples:
Does it depend on short interviews or extended trial work with current core members of the org? If the former, the company culture will be eroded quickly, and if the latter, much less likely.
Does the hiring process heavily use the contacts of the current employees? Using top recommendations from existing teammates can quickly zoom in on those with the most appropriate competencies and thinking patterns.
For intellectual traditions, I’d look to how much selection the practitioners of the art had on who followed them, and whether they were able to train them a great deal. For example, if you’re trying to continue a tradition of building compilers a certain way at Stanford, if you’re a professor there you’ll get to choose your postdocs from a wide set of excellent PhD graduates. If you’re able to work with a number of them first too, then you get a really strong sense of how they all think. An extra fun thing is that you’re able (with such a technical field as CS) to pick technical problems that you give people when they apply, and that you swear them to secrecy on, so that nobody can learn them in advance and practice for them. Pick a question that’s a good indicator of the core technical understanding, and you’re off to the races.
Any process of passing on knowledge that doesn’t have these attributes will be correspondingly worse.
Another thought on this is from HPMOR. As it says in the book, the difference between Minister for Magic, and Order of Merlin is that successive Ministers are voted in by the wizarding public, whereas the Order of Merlin is handed on by personal choice of the previous holder of the Order of Merlin at their death—and this is why the Minister of Magic is Cornelius Fudge, and the Order of Merlin is Albus Dumbledore. An unbroken line from Merlin himself. This is related to the students of Noble Prize Winners often being incredible themselves (something I’ve been told but never verified myself FYI).
So these are hiring analogies. I expect that most of Samo’s examples will be natural to this framework. If not… then it’s probably due to looking at different scales of the process. For example, you can guess if a company has a good hiring process if they’re successful, so you can look back at great companies and suppose they’re successful. Analogously, if a field (e.g. mathematics) consistently produces great insights over the course of a century, you can guess that the tradition was alive. And at the other end of the scale, you can probably look at the general rationality signs of the person doing the hiring, as opposed to the process itself.
Okay, that’s my brainstorming.
goes and reads post
Thoughts:
I notice I had thought some of the points, and dismissed them. For example having shared methodology, concepts or conceptual framework seemed to me too easy to goodhart on, so I dismissed them.
However, (a) they are still bayesian evidence toward the tradition being passed on correctly, and (b) if there is little incentive for people to game the system, then they’re perfectly good metrics.
I passingly mentioned judging output, and I agree with the OP that this is in fact the strongest signal. (I used a similar one myself here.)
Existence of a physical location did not even slightly enter my mind. But it totally fits the company framing—I’ve worked on remote teams before and they’re terrible at producing quality work together and transferring insights to each other.
Overall I had an okay basic framework, but the post had a delightful amount of detail that I either hadn’t noticed or hadn’t made explicit, so I give myself a C+ grade. (And now I’m super excited for sequels to this post!)
Excellent exercise! It seems a good way to evaluate content, initially building out or explicating one’s own models before comparing it to someone else’s, I will remember to do this.
I’m happy to hear you’ve found the model I present on useful and interesting!
Thanks. The important parts I neglected to do (due to time, will see if I can come back and finish in next few days) are:
Figure out what explicitly to add to my thinking that would’ve given me an A+. What simple models was I missing?
Ask if there was any way I could’ve figured it out from first principles in advance, without reading your post and making the update in part 1.