The point being that actually, it is worthwhile to point out simply that it is not a logical necessity—because people actually believe that. Once that is accepted, it clears the way for discussion of the actual probability that the AI does such a good job.
Therefore there is not one thing wrong with the excerpt that you quoted (and if you have a problem with another part of the essay, you should at least point out where the fallacy is).
The point being that actually, it is worthwhile to point out simply that it is not a logical necessity—because people actually believe that. Once that is accepted, it clears the way for discussion of the actual probability that the AI does such a good job.
Therefore there is not one thing wrong with the excerpt that you quoted (and if you have a problem with another part of the essay, you should at least point out where the fallacy is).