I think almost any coherent vision of a good future involves some kind of pause / restriction on unsafe AI development. We should probably not wait with preparing it for when AI labour is directed to it during takeoff.
I am dissatisfied / frustrated with how many leading figures in the AI industry (Altman, Musk, Huang, etc) do not match a simple query to how I’d image an epistemically humble responsible person would act and communicate. Primarily by not talking about catastrophic risk as a serious possibility worth preparing for.
I haven’t thought about what mechanisms exist to relay this information to governance processes and am unsure to what extent functional ones that could address this even exist which is why I’m putting it in my quick takes for now
There’s approximately 3 attitudes towards AGI Safety:
Dismissal from people that don’t take the possibility powerful AI seriously
Lip service, but motivational apathy from people that agree it’s epistemically sensible to take powerful AI seriously as a far mode sort of idea, but have other priorities they care much more about
Concern and caring from people that take powerful AI seriously
I encounter 2. a bunch as a local group organiser. I think often it’s because people think there’s too much uncertainty or too little leverage they have such that they’re better off focusing their motivation on worlds without future powerful AI. Seeing that experts take powerful AI seriously through podcasts etc and talking with other people at conferences that do seems to shift people towards 3. sometimes.
I think 3. maybe has some more interesting distinctions between:
3.1 People that assume alignment is easy to medium difficult and organisations will act pretty competently
3.2 People that take an uncertain attitude about alignment difficulty and organisational competence
3.3 People that assume alignment is difficult and organisations will act pretty incompetently
I think it’d be good for the world to have more people with the 3.2 attitude. Maybe a good example of people that I think of as representative are Buck Shlegeris and Ryan Greenblatt.
My intuition is that 3.3 gaining more traction would be good for the world, because to me a) that seems most realistic based on the evidence I’ve seen so far and how I interpret it, and b) least problematic in case it’s wrong and we live in a world where alignment isn’t hard and organisations act competently. What reasons make your intuition point towards 3.2?
Generally I think it’s good for people to try to be as accurate as possible and that’s what I most associate with 3.2. That said these clusters are a big oversimplification and I think there’s people with good reasoning that end up in 3.3. And in practice 3.3 might be a reasonable attitude just from a perspective of a sane society wanting to address the alignment is hard and orgs are incapable scenario anyway
The only place I expect an organisation with excellent safety culture, security mindset and strategic philosophical competence capable of steering a singularity towards good outcomes to arise is in a data center housing a country of aligned roughly human level AIs.
I think almost any coherent vision of a good future involves some kind of pause / restriction on unsafe AI development. We should probably not wait with preparing it for when AI labour is directed to it during takeoff.
I am dissatisfied / frustrated with how many leading figures in the AI industry (Altman, Musk, Huang, etc) do not match a simple query to how I’d image an epistemically humble responsible person would act and communicate. Primarily by not talking about catastrophic risk as a serious possibility worth preparing for.
I haven’t thought about what mechanisms exist to relay this information to governance processes and am unsure to what extent functional ones that could address this even exist which is why I’m putting it in my quick takes for now
There’s approximately 3 attitudes towards AGI Safety:
Dismissal from people that don’t take the possibility powerful AI seriously
Lip service, but motivational apathy from people that agree it’s epistemically sensible to take powerful AI seriously as a far mode sort of idea, but have other priorities they care much more about
Concern and caring from people that take powerful AI seriously
I encounter 2. a bunch as a local group organiser. I think often it’s because people think there’s too much uncertainty or too little leverage they have such that they’re better off focusing their motivation on worlds without future powerful AI. Seeing that experts take powerful AI seriously through podcasts etc and talking with other people at conferences that do seems to shift people towards 3. sometimes.
I think 3. maybe has some more interesting distinctions between:
3.1 People that assume alignment is easy to medium difficult and organisations will act pretty competently
3.2 People that take an uncertain attitude about alignment difficulty and organisational competence
3.3 People that assume alignment is difficult and organisations will act pretty incompetently
I think it’d be good for the world to have more people with the 3.2 attitude. Maybe a good example of people that I think of as representative are Buck Shlegeris and Ryan Greenblatt.
My intuition is that 3.3 gaining more traction would be good for the world, because to me a) that seems most realistic based on the evidence I’ve seen so far and how I interpret it, and b) least problematic in case it’s wrong and we live in a world where alignment isn’t hard and organisations act competently. What reasons make your intuition point towards 3.2?
Generally I think it’s good for people to try to be as accurate as possible and that’s what I most associate with 3.2. That said these clusters are a big oversimplification and I think there’s people with good reasoning that end up in 3.3. And in practice 3.3 might be a reasonable attitude just from a perspective of a sane society wanting to address the alignment is hard and orgs are incapable scenario anyway
The only place I expect an organisation with excellent safety culture, security mindset and strategic philosophical competence capable of steering a singularity towards good outcomes to arise is in a data center housing a country of aligned roughly human level AIs.