This is such a profound idea. An LLM scaffold built explicitly on rationalist principles might be exactly what is needed to get LLMs (not even necessarily frontier ones) to scale to accurately thinking about complex problems and producing accurate conclusions on them.
Insofar as there are similarities in cognitive biases between humans and LLMs (and there probably are many similarities, since e.g. biases like motivated thinking arise from the need to prioritize thinking time to different topics in general), the same reasoning techniques that let humans think better should also work on LLMs. Hence, prompts, scaffolds, and fine-tunings that teach LLMs explicitly to follow rationalist principles seem to me like a worthwhile area of investigation.
> Rationalist LLM systems for research
This is such a profound idea. An LLM scaffold built explicitly on rationalist principles might be exactly what is needed to get LLMs (not even necessarily frontier ones) to scale to accurately thinking about complex problems and producing accurate conclusions on them.
Insofar as there are similarities in cognitive biases between humans and LLMs (and there probably are many similarities, since e.g. biases like motivated thinking arise from the need to prioritize thinking time to different topics in general), the same reasoning techniques that let humans think better should also work on LLMs. Hence, prompts, scaffolds, and fine-tunings that teach LLMs explicitly to follow rationalist principles seem to me like a worthwhile area of investigation.