I think in short timelines we should relatively deprioritise longer term research bets because:
Clearly these bets are strictly better in long timelines where you both get the AI R&D acceleration and the time leading up to this.
It’s unclear how useful work done in advance will be for providing a head start to AIs automating AI R&D. E.g., maybe if things go well these AIs reinvent all prior progress very quickly and so the key thing is getting the meta-level process of this automation to work. One key question is whether the earlier work is key for supervising AIs or helps get tons of value from relatively weak AIs.
It’s unclear when automated AI R&D yields large accelerates relative to when risk emerges and AIs might be scheming and trying to sandbag at the first point when they would otherwise be very useful. (So, easier to set up methods which aims to ensure these AIs don’t sabotage work seems relatively more useful in short timelines.)
I think you probably agree that short timelines make longer term research bets some amount less useful (as you noted some other agendas become more important in short timelines). So, this is ultimately a quantitative question. I feel tempted by a perspective in which research that fully depends on massive serial acceleration gets deprioritized by ~3x conditional on short timelines (maybe <3 years) relative to >10 year timelines due to these factors.
To the extent AIs are not already robustly aligned superintelligences, the priorities they might put on AI risk related projects of their own initiative might be suboptimal for our purposes. If humans already have their R&D priorities straight (based on previous humans-substantially-in-the-loop research), they might be able to keep the AIs working on the right things, even if the AIs don’t have sufficient propensity to go there spontaneously.
I think in short timelines we should relatively deprioritise longer term research bets because:
Clearly these bets are strictly better in long timelines where you both get the AI R&D acceleration and the time leading up to this.
It’s unclear how useful work done in advance will be for providing a head start to AIs automating AI R&D. E.g., maybe if things go well these AIs reinvent all prior progress very quickly and so the key thing is getting the meta-level process of this automation to work. One key question is whether the earlier work is key for supervising AIs or helps get tons of value from relatively weak AIs.
It’s unclear when automated AI R&D yields large accelerates relative to when risk emerges and AIs might be scheming and trying to sandbag at the first point when they would otherwise be very useful. (So, easier to set up methods which aims to ensure these AIs don’t sabotage work seems relatively more useful in short timelines.)
I think you probably agree that short timelines make longer term research bets some amount less useful (as you noted some other agendas become more important in short timelines). So, this is ultimately a quantitative question. I feel tempted by a perspective in which research that fully depends on massive serial acceleration gets deprioritized by ~3x conditional on short timelines (maybe <3 years) relative to >10 year timelines due to these factors.
To the extent AIs are not already robustly aligned superintelligences, the priorities they might put on AI risk related projects of their own initiative might be suboptimal for our purposes. If humans already have their R&D priorities straight (based on previous humans-substantially-in-the-loop research), they might be able to keep the AIs working on the right things, even if the AIs don’t have sufficient propensity to go there spontaneously.