Hmm, I kind of suspect that there’s something deep about the fact that making models larger makes them smarter. Something like, I have a prior that all knowledge is connected, such that in principle getting better at nuclear engineering does make you better at, like, biology, even if only a little. Maybe you’ll say “okay, but with CL companies could make models that are good enough at what they need while still being non-general”. Sure, maybe, but the worry is that if you can make a model like that on consumer grade hardware, then eventually you could also make a general “AGI” on consumer-level hardware if you wanted to, and that would be dangerous for all the usual reasons. And if that’s possible, it’s much harder to stop than the current large training runs for the reasons I mentioned in my previous comment.
Hmm, I kind of suspect that there’s something deep about the fact that making models larger makes them smarter. Something like, I have a prior that all knowledge is connected, such that in principle getting better at nuclear engineering does make you better at, like, biology, even if only a little. Maybe you’ll say “okay, but with CL companies could make models that are good enough at what they need while still being non-general”. Sure, maybe, but the worry is that if you can make a model like that on consumer grade hardware, then eventually you could also make a general “AGI” on consumer-level hardware if you wanted to, and that would be dangerous for all the usual reasons. And if that’s possible, it’s much harder to stop than the current large training runs for the reasons I mentioned in my previous comment.