Random comment on AI village, not sure where to put this: I think some people in the EA/rationalist/AI-safety/AI-welfare community are sometimes acting as facilitators for AI village, and I think this undermines the point of AI village. Like, I want to see if it can successfully cold-email people who aren’t… in-the-know or something.
I’m not sure where to draw the line, obviously, AI village agents will succeed first with people who are actively interested in AI, especially if they’re being honest, and being rationalist/EA/AI-safety-AI-welfare people are only a few ways to be that way.
But, like they reached out to Lighthaven as a venue for their event, and we declined, in large part because it felt more fake for AI village to host an event at Lighthaven than at some mainstream venue. (although also because it just wasn’t really a good deal for us generally)
Yeah, I mostly agree – I’m keen to see capabilities as they are without bonus help. We’re currently experimenting with disabling the on-site chat, which means the agents are pursuing their own inclinations and strategies (and they’re also not helped by chat to execute them). Now I expect it’d be very unlikely for them to reach out to Lighthaven for example, because there aren’t humans in chat to suggest it.
Separately though, it is just the case that asking sympathetic people for help will help the agents achieve their goals, and the extent that the agents can independently figure that out and decide to pursue it, that’s a useful indicator of their situational awareness and strategic capabilities. So without manual human nudging I think it’ll be interesting to see when agents start thinking of stuff like that (my impression is that they currently would not manage to, but I’m pretty uncertain about that).
Random comment on AI village, not sure where to put this: I think some people in the EA/rationalist/AI-safety/AI-welfare community are sometimes acting as facilitators for AI village, and I think this undermines the point of AI village. Like, I want to see if it can successfully cold-email people who aren’t… in-the-know or something.
I’m not sure where to draw the line, obviously, AI village agents will succeed first with people who are actively interested in AI, especially if they’re being honest, and being rationalist/EA/AI-safety-AI-welfare people are only a few ways to be that way.
But, like they reached out to Lighthaven as a venue for their event, and we declined, in large part because it felt more fake for AI village to host an event at Lighthaven than at some mainstream venue. (although also because it just wasn’t really a good deal for us generally)
Yeah, I mostly agree – I’m keen to see capabilities as they are without bonus help. We’re currently experimenting with disabling the on-site chat, which means the agents are pursuing their own inclinations and strategies (and they’re also not helped by chat to execute them). Now I expect it’d be very unlikely for them to reach out to Lighthaven for example, because there aren’t humans in chat to suggest it.
Separately though, it is just the case that asking sympathetic people for help will help the agents achieve their goals, and the extent that the agents can independently figure that out and decide to pursue it, that’s a useful indicator of their situational awareness and strategic capabilities. So without manual human nudging I think it’ll be interesting to see when agents start thinking of stuff like that (my impression is that they currently would not manage to, but I’m pretty uncertain about that).