I’m worried about high false positives if this is used for monitoring. Keywords that look dangerous can often also be explained by the model just considering that these are things someone might do, before deciding against it. By analogy, if you applied a J-lens to a human, then a policeman who is perfectly lawful would trigger a lot of keywords for criminal behavior all the time, because they notice opportunities that other people would use to commit crimes.
I’m worried about high false positives if this is used for monitoring. Keywords that look dangerous can often also be explained by the model just considering that these are things someone might do, before deciding against it. By analogy, if you applied a J-lens to a human, then a policeman who is perfectly lawful would trigger a lot of keywords for criminal behavior all the time, because they notice opportunities that other people would use to commit crimes.