You can filter out some of the cranks by checking the forecaster’s reasoning, data, credentials, and track record, by looking for a consensus of similarly-qualified people, and by taking the incentives of the forecasters into account. But this comes with its own problems:
To a non-expert, it’s hard to tell to what degree an expert’s area of specialization overlaps with the question at hand. Is a hospital administrator a trustworthy source of guidance on the risk that a novel coronavirus turns into a pandemic?
To a non-expert, it’s hard to tell whether an expert consensus is really what it seems, or whether it’s coalition-building by a political faction under the cloak of “objectivity.”
These are just a few examples.
In the end, you have to decide whether it’s easier to check the forecaster’s reasoning or their trustworthiness.
You can filter out some of the cranks by checking the forecaster’s reasoning, data, credentials, and track record, by looking for a consensus of similarly-qualified people, and by taking the incentives of the forecasters into account. But this comes with its own problems:
To a non-expert, it’s hard to tell to what degree an expert’s area of specialization overlaps with the question at hand. Is a hospital administrator a trustworthy source of guidance on the risk that a novel coronavirus turns into a pandemic?
To a non-expert, easy questions look hard, and hard questions sometimes look easy. Can we distinguish between the two?
To a non-expert, it’s hard to tell whether an expert consensus is really what it seems, or whether it’s coalition-building by a political faction under the cloak of “objectivity.”
These are just a few examples.
In the end, you have to decide whether it’s easier to check the forecaster’s reasoning or their trustworthiness.