It would make it more competitive with AI capabilities work, diminishing the massive incentive to do the one that kills us rather than the one that prevents the killing.
(Not that I endorse all safety projects as really being safety projects; nor all capabilities projects as being on-the-path-to-extinction capabilites projects.)
I reckon it’s a small relative effect on the bigger capabilities pool, but a big relative effect on the smaller safety pool, in terms of raising the level of talent it can compete for.
Oops, I read your comment as just saying “diminishing the massive incentive to do the one that kills us” and missed “rather than the one that prevents the killing”. Agree with small effect on capabilities pool, potentially big effect on safety pool.
It would make it more competitive with AI capabilities work, diminishing the massive incentive to do the one that kills us rather than the one that prevents the killing.
(Not that I endorse all safety projects as really being safety projects; nor all capabilities projects as being on-the-path-to-extinction capabilites projects.)
Surely this effect is tiny right? Like what fraction of capabilities researchers will plausibly change what they do?
I reckon it’s a small relative effect on the bigger capabilities pool, but a big relative effect on the smaller safety pool, in terms of raising the level of talent it can compete for.
Oops, I read your comment as just saying “diminishing the massive incentive to do the one that kills us” and missed “rather than the one that prevents the killing”. Agree with small effect on capabilities pool, potentially big effect on safety pool.