When humans skim semi contradictory text, they produce a more consistent world model that doesn’t quite match up with what is said.
I felt like something like this happened to me when I was reading some of the “nonsensical” examples in the post, rather than deeming the text outright nonsensical and non-human I just interpreted it as the writer being sloppy.
Me too. I found it strongly reminiscent of reading low grade click bait. Or trying to listen to some woo. Part of it feels like rescuing some food when the bottom of the pan is burned. Part of it is like throwing out models of what state the author’s head was in that resolve the text into sense.
I think what makes GPT2 look relatively good is how low the baseline is in many respects. If you tell me ‘this is a political acceptance speech’ I don’t actually expect it to make that much sense. Most of the genre seems to be written by autocomplete anyway.
I felt like something like this happened to me when I was reading some of the “nonsensical” examples in the post, rather than deeming the text outright nonsensical and non-human I just interpreted it as the writer being sloppy.
Me too. I found it strongly reminiscent of reading low grade click bait. Or trying to listen to some woo. Part of it feels like rescuing some food when the bottom of the pan is burned. Part of it is like throwing out models of what state the author’s head was in that resolve the text into sense.
I think what makes GPT2 look relatively good is how low the baseline is in many respects. If you tell me ‘this is a political acceptance speech’ I don’t actually expect it to make that much sense. Most of the genre seems to be written by autocomplete anyway.