Empirically, AI progress seems to have been very continuous though rapid, and the gaps between model releases are smaller due to model releases happening more frequently. I don’t discount the possibility of model capabilities becoming very discontinuous in a few years, but it seems unlikely to me. This makes me pretty happy.
Also, it seems like doing alignment without a measurable goal is a difficult task for humans too. We might be a little better at estimating intermediate rewards but I think measuring alignment is necessary on both fronts
Empirically, AI progress seems to have been very continuous though rapid, and the gaps between model releases are smaller due to model releases happening more frequently. I don’t discount the possibility of model capabilities becoming very discontinuous in a few years, but it seems unlikely to me. This makes me pretty happy.
Also, it seems like doing alignment without a measurable goal is a difficult task for humans too. We might be a little better at estimating intermediate rewards but I think measuring alignment is necessary on both fronts