I would have wanted more pointing out of institutional capacity as part of the solution to this but I think it is a very good way of describing a more generalised re-focus to not goodharting on sub-parts of the problem.
Now, that I’ve said something serious I can finally comment on what I wanted to comment on:
Thanks to Ted Chiang, Toby Ord, and Hannu Rajaniemi for conversations which improved this piece.
Ted Chiang!!! I’m excited for some banger sci-fi based on things more relating to x-risk scenarios, that is so cool!
Let’s all keep in mind that the acknowledgement only says that Chiang and Rajaniemi had conversations with the author (Nielsen), and that Nielsen found those conversations helpful. For all we know, Chiang and Rajaniemi would strongly disagree with every word of this OP essay. If they’ve even read it.
See my other comment. I find it distressing that multiple people here are evidently treating acknowledgements as implying that the acknowledged person endorses the end product. I mean, it might or might be true in this particular case, but the acknowledgement is no evidence either way.
(For my part, I’ve taken to using the formula “Thanks to [names] for critical comments on earlier drafts”, in an attempt to preempt this mistake. Not sure if it works.)
I would have wanted more pointing out of institutional capacity as part of the solution to this but I think it is a very good way of describing a more generalised re-focus to not goodharting on sub-parts of the problem.
Now, that I’ve said something serious I can finally comment on what I wanted to comment on:
Ted Chiang!!! I’m excited for some banger sci-fi based on things more relating to x-risk scenarios, that is so cool!
Yeah being excited that Chiang and Rajaniemi are on board was one of my reactions to this excellent piece.
If you haven’t read Quantum Thief you probably should.
Let’s all keep in mind that the acknowledgement only says that Chiang and Rajaniemi had conversations with the author (Nielsen), and that Nielsen found those conversations helpful. For all we know, Chiang and Rajaniemi would strongly disagree with every word of this OP essay. If they’ve even read it.
I will check it out! Thanks!
Oh wow that’s surprising, I thought Ted Chiang was an AI skeptic not too long ago?
See my other comment. I find it distressing that multiple people here are evidently treating acknowledgements as implying that the acknowledged person endorses the end product. I mean, it might or might be true in this particular case, but the acknowledgement is no evidence either way.
(For my part, I’ve taken to using the formula “Thanks to [names] for critical comments on earlier drafts”, in an attempt to preempt this mistake. Not sure if it works.)