The more sacrifices someone has made, the easier it is to believe that they mean what they say.
Kokotajlo gave up millions to say what he wants, so I trust he is earnest. People who have gotten arrested at Stop AI have spent time in jail for their beliefs, so I trust they are earnest.
It doesn’t mean these people are most useful for AI safety but on the subject of trust I know no better measurement than sacrifice.
FVelde
Karma: 22
Stop AI discovered that the bill has been ‘amended’ to be entirely about aircraft liens now. See the following post.
https://x.com/StopAI_Info/status/1908914795308573108
The open letter now includes a postscript expressing their shock at the change.
The title, previously ‘Is taking extinction risk reasonable?’ has been changed to ‘On extinction risk over time and AI’. I appreciate the correction.
I agree that AI changes the likelihood of extinction rather than bring a risk where there was none before. In that sense the right question could be ‘Is increasing the probability of extinction reasonable?’.
Assuming that you mean by the last sentence that AI does not bring new extinction paths, I would like to counter that AI could well bring new paths to extinction; that is, there are probably paths to human extinction that open up when a machine intelligence surpasses human intelligence. Just like chess engines can apply strategies that humans have not thought of, some machine intelligence could find ways to wipe out humanity that have not yet been imagined. Furthermore, there might be ways to cause human extinction that can only be executed by a superior intelligence. An example could be a path that starts with hacking into many well defended servers in short succession, at a speed that even the best group of human hackers could not execute, in order to shut down a large part of the internet.
A lot of this seems to be pointing to ‘love’.