There were more vaguely concerning developments, but certainly nothing like “this is it, we’ve cracked sota using [X] online learning architecture”. Overall, my updates are towards LLMs + long context being sufficient for pretty dangerous capabilities, but with a decent chunk of room left for online learning to do a sudden leap basically out of nowhere (mostly because its so cheap to train one of these models).
Hey, sorry for the late reply. Quick summary:
There were more vaguely concerning developments, but certainly nothing like “this is it, we’ve cracked sota using [X] online learning architecture”. Overall, my updates are towards LLMs + long context being sufficient for pretty dangerous capabilities, but with a decent chunk of room left for online learning to do a sudden leap basically out of nowhere (mostly because its so cheap to train one of these models).