One is that many participants at ARENA have already done AI Safety research before participating. Second evidence is that at least four ARBOx (a 2-week compressed version of ARENA) are doing elite AI safety fellowships (1 Anthopic Fellows Program, 2 LASR Labs, 1 MATS).
I don’t think this is evidence that ARENA is about signalling:
MATS and LASR are completely different from ARENA. I wouldn’t find anything fishy about someone doing ARENA after LASR/MATS.
If anything, someone doing ARENA after MATS is evidence against the hypothesis, because the signal of (MATS+ARENA) is no better than MATS alone.
It sounds a little odd to do ARBOx after Anthropic Fellows. But I’d guess they wanted a fun two weeks hanging out in Oxford with other young people. Not that they thought the signal of (AFP + ARBOx) was so much better than AFP alone (which sounds absurd to me).
On 3., I think the four ARBOx fellows cited above did ARBOx first, then went on to AFP etc. I understand the argument to be “despite ARBOx being only two weeks, it has a good placement record, so why is ARENA 2.5x longer?”
1 and 2. This is subjective and more a gut feeling, but I think doing ARENA after having done LASR or MATS is not a good use of time, especially against the counter-factual of doing 4 research sprints. In my mind (and without more context), doing MATS and then ARENA would be a counter-signal—“How come you need to ARENA after having done MATS?”
I don’t think this is evidence that ARENA is about signalling:
MATS and LASR are completely different from ARENA. I wouldn’t find anything fishy about someone doing ARENA after LASR/MATS.
If anything, someone doing ARENA after MATS is evidence against the hypothesis, because the signal of (MATS+ARENA) is no better than MATS alone.
It sounds a little odd to do ARBOx after Anthropic Fellows. But I’d guess they wanted a fun two weeks hanging out in Oxford with other young people. Not that they thought the signal of (AFP + ARBOx) was so much better than AFP alone (which sounds absurd to me).
On 3., I think the four ARBOx fellows cited above did ARBOx first, then went on to AFP etc. I understand the argument to be “despite ARBOx being only two weeks, it has a good placement record, so why is ARENA 2.5x longer?”
1 and 2. This is subjective and more a gut feeling, but I think doing ARENA after having done LASR or MATS is not a good use of time, especially against the counter-factual of doing 4 research sprints. In my mind (and without more context), doing MATS and then ARENA would be a counter-signal—“How come you need to ARENA after having done MATS?”
3. See James Lester’s reply.