This post (and the author’s comments) don’t seem to be getting a great response and I’m confused why? The post seems pretty reasonable and the author’s comments are well informed.
My read of the main thrust is “don’t concentrate on a specific paradigm and instead look at this trend that has held for over 100 years”.
Can someone concisely explain why they think this is misguided? Is it just concerns over the validity of fitting parameters for a super-exponential model?
(I would also add that on priors when people claim “There is no way we can improve FLOPs/$ because of reasons XYZ” they have historically always been wrong.)
I think it’s good that someone is bringing this up. I think as a community we want to be deliberate and thoughtful with this class of things.
That being said, my read is that the main failure mode with advocacy at the moment isn’t “capabilities researchers are having emotional responses to being called out which is making it hard for them to engage seriously with x-risk.”
It’s “they literally have no idea that anyone thinks what they are doing is bad.”
Consider FAIR trying their hardest to open-source capabilities work with OPT. The tone and content of the responses shows overwhelming support for doing something that is, in my worldview, really, really bad.
I would feel much better if these people at least glanced their eyeballs over arguments for not open-source capabilities. Using the names of specific labs surely makes it more likely that the relevant writing ends up in front of them?