This seems basically reasonable, but as stated I think importantly misses that the plan you follow will change the accuracy of your estimates in steps (1) and (3) when you come to reassess. With 100% credence on some year, there’s no value in picking a plan that gets you evidence about timelines, or evidence of your likely impact in scenarios you’re assuming won’t happen.
It’s not enough to revisit the plan often if the plan you’re following isn’t giving you much new evidence.
Explore vs. exploit is a frame I naturally use (Though I do like your timeline-argmax frame, as well), where I ask myself “Roughly how many years should I feel comfortable exploring before I really need to be sitting down and attacking the hard problems directly somehow”?
Admittedly, this is confounded a bit by how exactly you’re measuring it. If I have 15-year timelines for median AGI-that-can-kill-us (which is about right, for me) then I should be willing to spend 5-6 years exploring by the standard 1/e algorithm. But when did “exploring” start? Obviously I should count my last eight months of upskilling and research as part of the exploration process. But what about my pre-alignment software engineering experience? If so, that’s now 4⁄19 years spent exploring, giving me about three left. If I count my CS degree as well, that’s 8⁄23 and I should start exploiting in less than a year.
Another frame I like is “hill-climbing”—namely, take the opportunity that seems best at a given moment. Though it is worth asking what makes something the best opportunity if you’re comparing, say, maximum impact now vs. maximum skill growth for impact later.
This seems basically reasonable, but as stated I think importantly misses that the plan you follow will change the accuracy of your estimates in steps (1) and (3) when you come to reassess. With 100% credence on some year, there’s no value in picking a plan that gets you evidence about timelines, or evidence of your likely impact in scenarios you’re assuming won’t happen.
It’s not enough to revisit the plan often if the plan you’re following isn’t giving you much new evidence.
Seems right. Explore vs. exploit is another useful frame.
Explore vs. exploit is a frame I naturally use (Though I do like your timeline-argmax frame, as well), where I ask myself “Roughly how many years should I feel comfortable exploring before I really need to be sitting down and attacking the hard problems directly somehow”?
Admittedly, this is confounded a bit by how exactly you’re measuring it. If I have 15-year timelines for median AGI-that-can-kill-us (which is about right, for me) then I should be willing to spend 5-6 years exploring by the standard 1/e algorithm. But when did “exploring” start? Obviously I should count my last eight months of upskilling and research as part of the exploration process. But what about my pre-alignment software engineering experience? If so, that’s now 4⁄19 years spent exploring, giving me about three left. If I count my CS degree as well, that’s 8⁄23 and I should start exploiting in less than a year.
Another frame I like is “hill-climbing”—namely, take the opportunity that seems best at a given moment. Though it is worth asking what makes something the best opportunity if you’re comparing, say, maximum impact now vs. maximum skill growth for impact later.