I think the core argument is “if you want to slow down, or somehow impose restrictions on AI research and deployment, you need some way of defining thresholds. Also, most policymaker’s cruxes appear to be that AI will not be a big deal, but if they thought it was going to be a big deal they would totally want to regulate it much more. Therefore, having policy proposals that can use future eval results as a triggering mechanism is politically more feasible, and also, epistemically helpful since it allows people who do think it will be a big deal to establish a track record”.
I find these arguments reasonably compelling, FWIW.
I think it would be good for more people to explicitly ask political staffers and politicians the question: “What hypothetical eval result would change your mind if you saw it?”
I think a lot of the evals are more targeted towards convincing tech workers than convincing politicians.
My sense is political staffers and politicians aren’t that great at predicting their future epistemic states this way, and so you won’t get great answers for this question. I do think it’s a really important one to model!
I think the core argument is “if you want to slow down, or somehow impose restrictions on AI research and deployment, you need some way of defining thresholds. Also, most policymaker’s cruxes appear to be that AI will not be a big deal, but if they thought it was going to be a big deal they would totally want to regulate it much more. Therefore, having policy proposals that can use future eval results as a triggering mechanism is politically more feasible, and also, epistemically helpful since it allows people who do think it will be a big deal to establish a track record”.
I find these arguments reasonably compelling, FWIW.
I think it would be good for more people to explicitly ask political staffers and politicians the question: “What hypothetical eval result would change your mind if you saw it?”
I think a lot of the evals are more targeted towards convincing tech workers than convincing politicians.
My sense is political staffers and politicians aren’t that great at predicting their future epistemic states this way, and so you won’t get great answers for this question. I do think it’s a really important one to model!
I believe the actual answer is “when it starts automating everything in the real world.”