Black box/outside view of the expert: This type of modelling would be just looking at the expert’s prediction accuracy in the past without asking about detailed properties of how they come to those decisions. Their prediction accuracy is ultimately what we want to get at but sometimes track records are incomplete or don’t exist yet.
[Worked out how to exit quote mode Pressing alt-enter 3 times works, today at least.]
You can do a lot better than this. Some signs of an expert from an outside perspective
1. Can predict the future better than simple extrapolations. 2. Can fix broken things better than everyman. 3. Can design and make things better than everyman. 4. Can explain things in a parsimonious way better than everyman,
All of the above need to take into account the possibility that luck played a part. For example if millions of people play the stock market and 29 get rich, then you need to take the large number of “attempts” in deciding whether the 29 have skill.
When you take this seriously it is astonishing now many ‘experts’ appear to have no skill at all.
You can do a lot better than this. Some signs of an expert from an outside perspective
1. Can predict the future better than simple extrapolations.
2. Can fix broken things better than everyman.
3. Can design and make things better than everyman.
4. Can explain things in a parsimonious way better than everyman,
All of the above need to take into account the possibility that luck played a part. For example if millions of people play the stock market and 29 get rich, then you need to take the large number of “attempts” in deciding whether the 29 have skill.
When you take this seriously it is astonishing now many ‘experts’ appear to have no skill at all.