I’m surprised to hear that you aren’t concerned about negative benchmarks being hillclimb targets for anyone. This updates me somewhat, though the hypotheses I’m still worried about are ones where the dishonest labs, whichever those turn out to be, are the main source of optimizing-for-bad-behavior-benchmarks. I also expect that bio/chem tasks that aren’t malicious-use-specific, which is the topic at hand, will get optimized for by less-dishonest labs, in at least some cases.
I’m surprised to hear that you aren’t concerned about negative benchmarks being hillclimb targets for anyone. This updates me somewhat, though the hypotheses I’m still worried about are ones where the dishonest labs, whichever those turn out to be, are the main source of optimizing-for-bad-behavior-benchmarks. I also expect that bio/chem tasks that aren’t malicious-use-specific, which is the topic at hand, will get optimized for by less-dishonest labs, in at least some cases.
Yeah, I feel much better about malicious use specific ones. Agreed that HLE is more generic and this is much worse