Where does the post mention Paul Christiano? I only see a link to a discussion, without any commentary.
Edit: Nvm, I figured it out. I assume you mean: “The general pattern is that one person or group argues that we know enough about a topic’s foundation that it’s time to start to focus on achieving near-term milestones, often engineering ones. ” is the specific line that you think doesn’t accurately capture your views.
Can you be more specific? If you help me understand how/if I’m misrepresenting, I’d be happy to change it. My sense is that Paul’s view is more like, “through working towards prosaic alignment, we’ll get a better understanding of whether there are insurmountable obstacles to alignment of scaled up (and likely better) models.” I can rephrase to something like that or something when more nuanced. I’m just wary of adding too much alignment-specific discussion as I don’t want the debate to be too focused on the object-level alignment debate.
It’s also worth noting that there are other researchers who hold similar views, so I’m not just talking about Paul’s.
FYI: I’ve updated the post to not talk about alignment at all, since I think focusing only on bottlenecks to progress in terms of capabilities makes the post clearer. Thanks to ChristianKI for pointing this out.
John_Maxwell_IV, would love feedback on how you feel about the edited version.
I think your original phrasing made it sound kinda like I thought that we should go full steam ahead on experimental/applied research. I agree with MIRI that people should be doing more philosophical/theoretical work related to FAI, at least on the margin. The position I was taking in the thread you linked was about the difficulty of such research, not its value.
With regard to the question itself, Christian’s point is a good one. If you’re solely concerned with building capability, alternating between theory and experimentation, or even doing them in parallel, seems optimal. If you care about safety as well, it’s probably better to cross the finish line during a “theory” cycle than an “experimentation” cycle.
FWIW, I can’t speak for Paul Christiano, but insofar as you’ve attempted to summarize what I think here, I don’t endorse the summary.
Where does the post mention Paul Christiano? I only see a link to a discussion, without any commentary.
Edit: Nvm, I figured it out. I assume you mean: “The general pattern is that one person or group argues that we know enough about a topic’s foundation that it’s time to start to focus on achieving near-term milestones, often engineering ones. ” is the specific line that you think doesn’t accurately capture your views.
Can you be more specific? If you help me understand how/if I’m misrepresenting, I’d be happy to change it. My sense is that Paul’s view is more like, “through working towards prosaic alignment, we’ll get a better understanding of whether there are insurmountable obstacles to alignment of scaled up (and likely better) models.” I can rephrase to something like that or something when more nuanced. I’m just wary of adding too much alignment-specific discussion as I don’t want the debate to be too focused on the object-level alignment debate.
It’s also worth noting that there are other researchers who hold similar views, so I’m not just talking about Paul’s.
FYI: I’ve updated the post to not talk about alignment at all, since I think focusing only on bottlenecks to progress in terms of capabilities makes the post clearer. Thanks to ChristianKI for pointing this out.
John_Maxwell_IV, would love feedback on how you feel about the edited version.
I think your original phrasing made it sound kinda like I thought that we should go full steam ahead on experimental/applied research. I agree with MIRI that people should be doing more philosophical/theoretical work related to FAI, at least on the margin. The position I was taking in the thread you linked was about the difficulty of such research, not its value.
With regard to the question itself, Christian’s point is a good one. If you’re solely concerned with building capability, alternating between theory and experimentation, or even doing them in parallel, seems optimal. If you care about safety as well, it’s probably better to cross the finish line during a “theory” cycle than an “experimentation” cycle.