Just in case it’s not, obvious. I think, people are reacting to the lack of caution and paranoia described in the testing document.
The subtext is that if anyone is going to take this seriously, it should be the people involved in ARC, since it’s so closely connected to lesswrong and EA. It’s the ingroup! It’s us! In other words: there are higher expectations on ARC than there are on Microsoft, this is because we should care the most. We’ve read the most science fiction, and spend decades of our lives arguing about it, after all.
Yet it doesn’t sound like testing was taken seriously at all, there was no security mindset displayed (if this is miscommunication, then please correct me).
If even we, who have spent many years caring, cannot be careful… then we all die but with no dignity points.
big_yud_screaming.jpeg
EDIT: if anyone is curious about how paranoid ARC is being… they haven’t told us. But they show a little of their workflowin this job ad. And it looks like a human copies each response manually, or executes each command themselves. This is what they mean by closely monitored.
Just in case it’s not, obvious. I think, people are reacting to the lack of caution and paranoia described in the testing document.
The subtext is that if anyone is going to take this seriously, it should be the people involved in ARC, since it’s so closely connected to lesswrong and EA. It’s the ingroup! It’s us! In other words: there are higher expectations on ARC than there are on Microsoft, this is because we should care the most. We’ve read the most science fiction, and spend decades of our lives arguing about it, after all.
Yet it doesn’t sound like testing was taken seriously at all, there was no security mindset displayed (if this is miscommunication, then please correct me).
If even we, who have spent many years caring, cannot be careful… then we all die but with no dignity points.
big_yud_screaming.jpeg
EDIT: if anyone is curious about how paranoid ARC is being… they haven’t told us. But they show a little of their workflow in this job ad. And it looks like a human copies each response manually, or executes each command themselves. This is what they mean by closely monitored.
EDIT2: see update from the authors