Imagine your hypothetical chain smoker lives in a world where there are so many chain smokers that the second hand smoke they experience accounts for a majority of their exposure to smoking-related health risks. If they unilaterally stop, they will still get lung cancer, but at age 68 instead of 65. But, for social reasons, they’ll then lose advancement opportunities in their careers, and lose opportunities to coordinate a society-wide smoking slowdown or restrictions on where people can smoke. And also that they care at least as much about their children, who don’t (yet) smoke, as themselves.
Edit to add: to be clear, I don’t think everyone is thinking in these more noble terms, and I’m sure many do have the aims described in the remainder of the post. But I think the analogy fails to capture important parts of the stiuational dynamics of AI
Imagine your hypothetical chain smoker lives in a world where there are so many chain smokers that the second hand smoke they experience accounts for a majority of their exposure to smoking-related health risks. If they unilaterally stop, they will still get lung cancer, but at age 68 instead of 65. But, for social reasons, they’ll then lose advancement opportunities in their careers, and lose opportunities to coordinate a society-wide smoking slowdown or restrictions on where people can smoke. And also that they care at least as much about their children, who don’t (yet) smoke, as themselves.
Edit to add: to be clear, I don’t think everyone is thinking in these more noble terms, and I’m sure many do have the aims described in the remainder of the post. But I think the analogy fails to capture important parts of the stiuational dynamics of AI