I’ve learned a lot from following the Agent Village, and I found this post quite persuasive. But I’m still uncertain whether it will have a positive impact, partly for the reasons you mentioned, but also for these reasons:
While agents are bad, the Agent village may simply convince people that agents are crappy and therefore harmless.
Once agents are good, AI companies will be falling over themselves to do demonstrations, making the Agent village obsolete, at least for raising awareness of agent capabilities.
On the first point, many people don’t seem to project capabilities forward. Possibly because they only pay enough attention to see snapshots, and so don’t appreciate the pace of progress. For many (most?) people, seeing a snapshot of Claude getting lost in excessive Google docs, or o3 forgetting it can’t talk, may simply make them believe agents are rubbish or even a dead end. We live in an AI bubble, and I suspect most people will only update if they see AIs successfully doing things, rather than failing to do things.
On the second point, would the Agent Village significantly change its messaging and strategy once AI agents are economically viable? Would they focus more on uncovering misalignment risks than evaluating capabilities?
On the second point, would the Agent Village significantly change its messaging and strategy once AI agents are economically viable? Would they focus more on uncovering misalignment risks than evaluating capabilities?
I hope and expect so yeah. In general I think that if they are just doing stuff that could easily have been a tech demo from a startup or bigco, they are doing it wrong. I’d like to see the AIs doing philosophy, making nonobvious ethical and political decisions, trying to forecast the future, trying to introspect on their values and their place in the world, trying to do good in the world (as opposed to make money), trying to be fully autonomous (as opposed to being a useful tool or assistant) …
I’ve learned a lot from following the Agent Village, and I found this post quite persuasive. But I’m still uncertain whether it will have a positive impact, partly for the reasons you mentioned, but also for these reasons:
While agents are bad, the Agent village may simply convince people that agents are crappy and therefore harmless.
Once agents are good, AI companies will be falling over themselves to do demonstrations, making the Agent village obsolete, at least for raising awareness of agent capabilities.
On the first point, many people don’t seem to project capabilities forward. Possibly because they only pay enough attention to see snapshots, and so don’t appreciate the pace of progress. For many (most?) people, seeing a snapshot of Claude getting lost in excessive Google docs, or o3 forgetting it can’t talk, may simply make them believe agents are rubbish or even a dead end. We live in an AI bubble, and I suspect most people will only update if they see AIs successfully doing things, rather than failing to do things.
On the second point, would the Agent Village significantly change its messaging and strategy once AI agents are economically viable? Would they focus more on uncovering misalignment risks than evaluating capabilities?
I hope and expect so yeah. In general I think that if they are just doing stuff that could easily have been a tech demo from a startup or bigco, they are doing it wrong. I’d like to see the AIs doing philosophy, making nonobvious ethical and political decisions, trying to forecast the future, trying to introspect on their values and their place in the world, trying to do good in the world (as opposed to make money), trying to be fully autonomous (as opposed to being a useful tool or assistant) …