I’ve changed my mind. Coming up with the ideas above has given me a better sense of how agent foundations progress could be valuable.
More concretely, I used to focus more on the ways in which agent foundations applied in the infinite-compute limit, which is not a setting that I think is very relevant. But I am more optimistic about agent foundations as the study of idealized multi-agent interactions. In hindsight a bunch of the best agent foundations research (e.g. logical induction) was both at once, but I’d only been viewing it as the former.
More abstractly, this update has made me more optimistic about conceptual progress in general. I guess it’s hard to viscerally internalize the extent to which it’s possible for concepts to break and reform in new configurations without having experienced that. (People who were strongly religious then deconverted have probably had a better sense of this all along, but I never had that experience.)
I’m glad I did the compromise before; it ended up reducing the consequences of this mistake somewhat (though I’d guess including one week of agent foundations in the course had a pretty small effect).
I’ve changed my mind. Coming up with the ideas above has given me a better sense of how agent foundations progress could be valuable.
More concretely, I used to focus more on the ways in which agent foundations applied in the infinite-compute limit, which is not a setting that I think is very relevant. But I am more optimistic about agent foundations as the study of idealized multi-agent interactions. In hindsight a bunch of the best agent foundations research (e.g. logical induction) was both at once, but I’d only been viewing it as the former.
More abstractly, this update has made me more optimistic about conceptual progress in general. I guess it’s hard to viscerally internalize the extent to which it’s possible for concepts to break and reform in new configurations without having experienced that. (People who were strongly religious then deconverted have probably had a better sense of this all along, but I never had that experience.)
I’m glad I did the compromise before; it ended up reducing the consequences of this mistake somewhat (though I’d guess including one week of agent foundations in the course had a pretty small effect).