Oxford AI Safety Initiative (President)
James Lester
Karma: 8
I’ve worked on admissions/strategy/evaluation for ARBOx2 and ARBOx3, and I’d broadly endorse JamesH’s comments—I’d be very excited to see the programme you describe, but as a next step/complement to ARENA-type programmes rather than a substitute.
A few thoughts on the signalling/upskilling conversation:
Based on surveys and private anecdata, it seems like a large part of ARBOx’s value comes from building confidence, motivation, and a sense of direction for participants. I think it’s hard to replicate this benefit through self-studying the ARENA curriculum, and I’d say it’s a significant component of ARBOx’s past/ongoing impact. This could explain ARBOx’s strong placement record despite a compressed runtime (since these benefits probably scale sub-linearly with more weeks).
Moreover, participants reported that ARBOx was helpful for them “developing technical skills in ML for AI safety” (avg 9.15/10 agreement, much higher than statements pertaining to perceived signalling benefits). Of course, different participants benefit from ARBOx in different ways, and end-of-programme self-reports can be misleading (we’re doing a six-month follow-up now), but I think the “signalling” story is at best incomplete.
I also don’t think the signalling story follows from many ARENA/ARBOx applicants having prior safety research experience. Eyeballing the survey data, participants with (more) prior safety research experience didn’t find ARBOx any less useful for developing their technical skills (if anything, the opposite is true, although there are too many confounders to infer anything meaningful).
A few thoughts on time-efficiency:
I do think ARBOx is pretty time-efficient, which is very important to a lot of our participants (e.g. it fits in the two weeks between New Year and Oxford term starting). I don’t dispute that a minority of ARENA notebooks comprise the majority of the curriculum’s value, and that many ARBOx participants get >40% of the value they’d get from ARENA in 40% of the time. For this reason, I’d be excited to see more people put on ARBOx-style programmes.
In some cases, though, participants aren’t so time-constrained, and gains per week isn’t really the thing you care about —it seems like most participants would upskill faster from 5 weeks of ARENA + a few week-long sprints than 2 weeks of ARBOx + 3 weeks of ??? + a few week-long sprints. For what it’s worth: 14⁄19 ARBOx2 survey respondents reported they were happy with its length, 5⁄19 would have preferred a longer programme, and nobody wanted a shorter programme.
On 3., I think the four ARBOx fellows cited above did ARBOx first, then went on to AFP etc. I understand the argument to be “despite ARBOx being only two weeks, it has a good placement record, so why is ARENA 2.5x longer?”