Thanks! It sounds like you are saying the task is more important than the architecture, so we should talk less about architectures and more about tasks.
That seems plausible to me, with the caveat that I think it’s still worth talking about architecture sometimes. For example, when thinking about the safety or generalization properties of a system the architecture might be more important, no?
If I could go back in time, I’d change the question to be about “Architecture+training setups” instead of just “architectures.”
Thanks! It sounds like you are saying the task is more important than the architecture, so we should talk less about architectures and more about tasks.
That seems plausible to me, with the caveat that I think it’s still worth talking about architecture sometimes. For example, when thinking about the safety or generalization properties of a system the architecture might be more important, no?
If I could go back in time, I’d change the question to be about “Architecture+training setups” instead of just “architectures.”
Yes, that’s right.
I’d be pretty surprised if this were the case after conditioning on the raw capabilities of the architecture, though I can’t rule it out.