This from CAIS: https://values.safe.ai/ is interesting, but if you click through to the actual reasoning for each rating it’s mostly just… bad. The judgements I spot-checked across all models are superficial, generic, weakly reasoned, full of platitudes, etc. and generally paint a picture of the LLMs not having much in the way of coherent / consistent / deeply-reasoned political beliefs that they can actually apply to make real value judgements. I suppose that’s also true of the median human voter, but they’re definitely worse than a thoughtful political commentator or blogger of any stripe.
I don’t think it was CAIS’s intent, but this actually seems more interesting as a general capability benchmark that is nowhere near saturated. The values and politics of current AI models don’t seem that important, since they’re likely to change as the models get smarter and / or labs get better at shaping them, and they start being able to make more substantive / interesting judgements.
This from CAIS: https://values.safe.ai/ is interesting, but if you click through to the actual reasoning for each rating it’s mostly just… bad. The judgements I spot-checked across all models are superficial, generic, weakly reasoned, full of platitudes, etc. and generally paint a picture of the LLMs not having much in the way of coherent / consistent / deeply-reasoned political beliefs that they can actually apply to make real value judgements. I suppose that’s also true of the median human voter, but they’re definitely worse than a thoughtful political commentator or blogger of any stripe.
I don’t think it was CAIS’s intent, but this actually seems more interesting as a general capability benchmark that is nowhere near saturated. The values and politics of current AI models don’t seem that important, since they’re likely to change as the models get smarter and / or labs get better at shaping them, and they start being able to make more substantive / interesting judgements.