On the second point, would the Agent Village significantly change its messaging and strategy once AI agents are economically viable? Would they focus more on uncovering misalignment risks than evaluating capabilities?
I hope and expect so yeah. In general I think that if they are just doing stuff that could easily have been a tech demo from a startup or bigco, they are doing it wrong. I’d like to see the AIs doing philosophy, making nonobvious ethical and political decisions, trying to forecast the future, trying to introspect on their values and their place in the world, trying to do good in the world (as opposed to make money), trying to be fully autonomous (as opposed to being a useful tool or assistant) …
I hope and expect so yeah. In general I think that if they are just doing stuff that could easily have been a tech demo from a startup or bigco, they are doing it wrong. I’d like to see the AIs doing philosophy, making nonobvious ethical and political decisions, trying to forecast the future, trying to introspect on their values and their place in the world, trying to do good in the world (as opposed to make money), trying to be fully autonomous (as opposed to being a useful tool or assistant) …