A couple advantages for AI intellectuals could be: - being able to rerun based on different inputs, see how their analysis changes function of those inputs - being able to view full reasoning traces (while also not the full story, probably more of the full story than what goes on with human reasoning, good intellectuals already try to share their process but maybe can do better/use this to weed out clearly bad approaches)
On “rerun based on different inputs”, this would work cleanly with AI forecasters. You can literally say, “Given that you get a news article announcing a major crisis X that happens tomorrow, what is your new probability on Y?” (I think I wrote about this a bit before, can’t find it right now).
A couple advantages for AI intellectuals could be:
- being able to rerun based on different inputs, see how their analysis changes function of those inputs
- being able to view full reasoning traces (while also not the full story, probably more of the full story than what goes on with human reasoning, good intellectuals already try to share their process but maybe can do better/use this to weed out clearly bad approaches)
Yep!
On “rerun based on different inputs”, this would work cleanly with AI forecasters. You can literally say, “Given that you get a news article announcing a major crisis X that happens tomorrow, what is your new probability on Y?” (I think I wrote about this a bit before, can’t find it right now).
I did write more about a full-scale forecasting system could be built and evaluated, here, for those interested:
https://www.lesswrong.com/posts/QvFRAEsGv5fEhdH3Q/preliminary-notes-on-llm-forecasting-and-epistemics
https://www.lesswrong.com/posts/QNfzCFhhGtH8xmMwK/enhancing-mathematical-modeling-with-llms-goals-challenges
Overall, I think there’s just a lot of neat stuff that could be done.