I was at a party recently, and happened to meet a senior person at a well-known AI startup in the Bay Area. They volunteered that they thought “humanity had about a 50% chance of extinction” caused by artificial intelligence. I asked why they were working at an AI startup if they believed that to be true. They told me that while they thought it was true, “in the meantime I get to have a nice house and car”.
This strikes me as the sort of thing one would say without quite meaning it. Like, I’m sure this person could get other jobs that also support a nice house and car. And if they thought about it, they could probably also figure this out. I’m tempted to chalk the true decision up to conformity / lack of confidence in one’s ability to originate and execute consequentialist plans, but that’s just a guess and I’m not particularly well-informed about this person.
To paraphrase Von Neumann, sometimes we confess to a selfish motive that we may not be suspected of an unselfish one, or to one sin to avoid being accused of another.
[Of] the splendid technical work of the [atomic] bomb there can be no question. I can see no evidence of a similar high quality of work in policy-making which...accompanied this...Behind all this I sensed the desires of the gadgeteer to see the wheels go round.
This strikes me as the sort of thing one would say without quite meaning it. Like, I’m sure this person could get other jobs that also support a nice house and car. And if they thought about it, they could probably also figure this out. I’m tempted to chalk the true decision up to conformity / lack of confidence in one’s ability to originate and execute consequentialist plans, but that’s just a guess and I’m not particularly well-informed about this person.
To paraphrase Von Neumann, sometimes we confess to a selfish motive that we may not be suspected of an unselfish one, or to one sin to avoid being accused of another.
(“as any number of conversations in the [OpenAI] office café will confirm, the “build AGI” bit of the mission seems to offer up more raw excitement to its researchers than the “make it safe” bit.”)
Or perhaps they thought it was an entertaining response and don’t actually believe in the fear narrative.