(brought here by Kaj Sotala’s comment) I’d be curious how you’ve updated on these takes. My own vague impressions: still bad at research taste, reliability has improved tremendously but still brittle in some sense, lots of progress solving non-templated problems especially in math but even there original seeing seems missing, still considerably diminishing returns on problem-solving time. Overall not really closing the gap to Byrnesian AGI, although maybe closing the gap to PASTA-AGI.
I’d tend to agree with your assessment. We are certainly in the “awkward intermediate period” I mention in my fast scenario. Overall, I think we’re closer to what I called the fast scenario than the slow scenario, but we’re definitely not in my full-on fast scenario. As you say, there is still a brittleness to model capabilities, lacking in taste and high-level judgement. The most impressive results, such as in mathematics, always seem to come with important asterisks – one gets the strong sense that the achievements are limited to certain kinds of problems and/or are missing important attributes such as a deep insight to accompany the proof.
(brought here by Kaj Sotala’s comment) I’d be curious how you’ve updated on these takes. My own vague impressions: still bad at research taste, reliability has improved tremendously but still brittle in some sense, lots of progress solving non-templated problems especially in math but even there original seeing seems missing, still considerably diminishing returns on problem-solving time. Overall not really closing the gap to Byrnesian AGI, although maybe closing the gap to PASTA-AGI.
I’d tend to agree with your assessment. We are certainly in the “awkward intermediate period” I mention in my fast scenario. Overall, I think we’re closer to what I called the fast scenario than the slow scenario, but we’re definitely not in my full-on fast scenario. As you say, there is still a brittleness to model capabilities, lacking in taste and high-level judgement. The most impressive results, such as in mathematics, always seem to come with important asterisks – one gets the strong sense that the achievements are limited to certain kinds of problems and/or are missing important attributes such as a deep insight to accompany the proof.