Suggested variation, which I’d expect to lead to better results: use raw “completion probabilities” for different answers.E.g. with prompt “Will Russia invade Ukrainian territory in 2022?” extract completion likelihoods of the next few tokes “Yes” and “No”. Normalize.(the prompts you ask for quite unnatural map, at which most untrained humans are pretty bad as well)
Current theme: default
Less Wrong (text)
Less Wrong (link)
Suggested variation, which I’d expect to lead to better results: use raw “completion probabilities” for different answers.
E.g. with prompt “Will Russia invade Ukrainian territory in 2022?” extract completion likelihoods of the next few tokes “Yes” and “No”. Normalize.
(the prompts you ask for quite unnatural map, at which most untrained humans are pretty bad as well)