It is just a useful assumption for control research. Not having to run evals on humans’ ability to audit transcripts speeds things up. For what it’s worth, I am truly a bit uncertain here and plan to run human-auditing experiments at some point.
Fair enough. Still, the industrial/organizational psychologists will be happy to tell you that, as far as they can determine, even the most competent humans are never perfect at anything, including tasks vastly simpler than this, regardless of the stakes involved.
It makes me sad that many people seem to, in fact, be uncertain about this and think it’s actually a plausible possibility.
It is just a useful assumption for control research. Not having to run evals on humans’ ability to audit transcripts speeds things up. For what it’s worth, I am truly a bit uncertain here and plan to run human-auditing experiments at some point.
Fair enough. Still, the industrial/organizational psychologists will be happy to tell you that, as far as they can determine, even the most competent humans are never perfect at anything, including tasks vastly simpler than this, regardless of the stakes involved.