This is not a relationship of “These guys are building systems that may cause humanity’s extinction and we must stop them.”, and it’s not even one of “There are clear standards that corporations must abide by, or else.”
It is one of “We are their subordinate and depend on access to their APIs. We hope that one day, our work will be useful in helping them not deploy dangerous systems. In the meantime, we de facto help with their PR.”
Even assuming your latter claim about AI evals orgs is entirely true, isn’t this enough to make evals organizations useful?
Any AI regulations that actually constrain AI companies must depend on an enforcement mechanism. For the lawmakers working on such legislation, it is convenient if such enforcement mechanism already exists and can simply be activated. Therefore, the existence of AI evals orgs makes it easier to pass legislation to constrain AI companies and improve their safety—with the orgs already in place and researching effective AI safety evals, the only thing the law has to effectively change is to force the AI labs to work with AI evals orgs rather than having it be voluntary. Isn’t this pretty much the same direction that those funding and working at these organizations today would hope for?
Even assuming your latter claim about AI evals orgs is entirely true, isn’t this enough to make evals organizations useful?
Any AI regulations that actually constrain AI companies must depend on an enforcement mechanism. For the lawmakers working on such legislation, it is convenient if such enforcement mechanism already exists and can simply be activated. Therefore, the existence of AI evals orgs makes it easier to pass legislation to constrain AI companies and improve their safety—with the orgs already in place and researching effective AI safety evals, the only thing the law has to effectively change is to force the AI labs to work with AI evals orgs rather than having it be voluntary. Isn’t this pretty much the same direction that those funding and working at these organizations today would hope for?