I think the special brain algorithms in question—e.g., the ones that make us comfortable entrusting a neurotypical human to decide who won in the set-up above—are more familiarly thought of as prosocial or moralcognition. A claim like this would predict that we would be uncomfortable entrusting humans who lacked the relevant prosocial instincts (e.g., psychopaths) to oversee a safety-via-debate-type set-up, which seems correct.
This passage seems to treat psychopaths as lacking prosocial circuitry causing uncomfortability entrusting. In the parent psychopaths seem to be members of those that do instantiate the circuitry. I am a bit confused.
For example some people could characterise asperger cognition as “asocial cognition”. Picking out one cognition as “moral” easily slips that anything that is not that is “immoral”.
I see the parent comment making a claim analogous in logical structure to “I did not say we distrust women as politicians. I just said that we trust men as politicians.”
I suppose the next post will explain it in greater detail.
I am not an expert, but it seems to me that moral blindness is the main thing the psychopaths are famous for. This is not about excluding a random group of people, but about noticing that something is not a universal human experience and pointing out the specific group that lacks it. It’s like: “if we want to improve the safety of a self-driving car, we could keep a human in the loop who will also observe the road and provide feedback to the AI—but of course we should not choose a blind human for this role”.
I think the reason for objecting to this portion is the specificity and massive over-simplification of the criterion for trust. There are LOTS of reasons not to trust someone, and neurodiversity is not actually one of them. Psychopathy probably is, but it’s really hard to test for, and it’s not exclusive. We may want to exclude religious or cultural true-believers, but they’re not called out here for some reason.
The question of what specific humans to entrust with such a thing is really hard, and important. Reducing it to “neurotypical” is simply incorrect.
I agree with this. There’s a lot going on and I think what we’re looking for isn’t (just) some kind of “prosocial algorithm” but rather a mind developed around integrity that probably comes with prosocial instincts and so on. In theory, we could imagine a self-aware psychopath who takes extreme care to avoid self-serving cognition and would always self-modify in prosocial ways. I’m not sure if such people exist, but I think it’s important not to blanket-dismiss an entire class of people with a specific set of chracteristics (even if the characteristics in question are “antisocial” [at least “system-1 antisocial”] by definition).
I broadly agree with Viliam’s comment above. Regarding Dagon’s comment (to which yours is a reply), I think that characterizing my position here as ‘people who aren’t neurotypical shouldn’t be trusted’ is basically strawmanning, as I explained in this comment. I explicitly don’t think this is correct, nor do I think I imply it is anywhere in this post.
As for your comment, I definitely agree that there is a distinction to be made between prosocial instincts and the learned behavior that these instincts give rise to over the lifespan, but I would think that the sort of ‘integrity’ that you point at here as well as the self-aware psychopath counterexample are both still drawing on particular classes of prosocial motivations that could be captured algorithmically. See my response to ‘plausible critique #1,’ where I also discuss self-awareness as an important criterion for prosociality.
This passage seems to treat psychopaths as lacking prosocial circuitry causing uncomfortability entrusting. In the parent psychopaths seem to be members of those that do instantiate the circuitry. I am a bit confused.
For example some people could characterise asperger cognition as “asocial cognition”. Picking out one cognition as “moral” easily slips that anything that is not that is “immoral”.
I see the parent comment making a claim analogous in logical structure to “I did not say we distrust women as politicians. I just said that we trust men as politicians.”
I suppose the next post will explain it in greater detail.
I am not an expert, but it seems to me that moral blindness is the main thing the psychopaths are famous for. This is not about excluding a random group of people, but about noticing that something is not a universal human experience and pointing out the specific group that lacks it. It’s like: “if we want to improve the safety of a self-driving car, we could keep a human in the loop who will also observe the road and provide feedback to the AI—but of course we should not choose a blind human for this role”.
I think the reason for objecting to this portion is the specificity and massive over-simplification of the criterion for trust. There are LOTS of reasons not to trust someone, and neurodiversity is not actually one of them. Psychopathy probably is, but it’s really hard to test for, and it’s not exclusive. We may want to exclude religious or cultural true-believers, but they’re not called out here for some reason.
The question of what specific humans to entrust with such a thing is really hard, and important. Reducing it to “neurotypical” is simply incorrect.
I agree with this. There’s a lot going on and I think what we’re looking for isn’t (just) some kind of “prosocial algorithm” but rather a mind developed around integrity that probably comes with prosocial instincts and so on. In theory, we could imagine a self-aware psychopath who takes extreme care to avoid self-serving cognition and would always self-modify in prosocial ways. I’m not sure if such people exist, but I think it’s important not to blanket-dismiss an entire class of people with a specific set of chracteristics (even if the characteristics in question are “antisocial” [at least “system-1 antisocial”] by definition).
I broadly agree with Viliam’s comment above. Regarding Dagon’s comment (to which yours is a reply), I think that characterizing my position here as ‘people who aren’t neurotypical shouldn’t be trusted’ is basically strawmanning, as I explained in this comment. I explicitly don’t think this is correct, nor do I think I imply it is anywhere in this post.
As for your comment, I definitely agree that there is a distinction to be made between prosocial instincts and the learned behavior that these instincts give rise to over the lifespan, but I would think that the sort of ‘integrity’ that you point at here as well as the self-aware psychopath counterexample are both still drawing on particular classes of prosocial motivations that could be captured algorithmically. See my response to ‘plausible critique #1,’ where I also discuss self-awareness as an important criterion for prosociality.
Yes, that section is overly simplistic but not in a way that couldn’t be fixed or would invalidate the thrust or the main argument.