There is ambiguity in “source of values” which this outline elides. There is source as “method of discovery” and source as “kind of fact that determines the truth value of normative statements” and source as “causal origin of value-based thinking and language”. And there is no reason why the “source of values” should be the same for each of those readings. Though obviously they could be.
So under “conscious cognition”: traditional Kantian deontology is discovered by conscious cognition but the facts that make it true are supposed to be external to human minds (purportedly they exist in reason itself, like mathematical truths). But there are definitely also people who hold the view that what makes a decision a moral one is just that it was arrived at by the properly and rationally reflecting on values.
I don’t think any position can be accurately categorized without understanding how that position answers each of the above readings of “source”. So for instance, my own position would be that the causal origin of values is a synthesis of culturally constructed intuitions and innate, evolved intuitions on which we perform conscious cognitive operations (which are themselves a synthesis of innate abilities and technologies). So a mix between conscious and unconscious cognition, other humans, and some degree of reason. I don’t think there are any facts that make value language true and method of discovery is just what we think we’re doing with the above. It’s then helpful to ask something like: well how do you want a superintelligence to decide what to do? The answer to which somehow involves recursing the causal source of value judgments upon itself. E.g. It’s the same as asking “How ought a super intelligence behave?” which is just an instance of a value judgment. But obviously it is really difficult to a) precisely define that process and b)determine how exactly that recursion ought to happen.
There is ambiguity in “source of values” which this outline elides. There is source as “method of discovery” and source as “kind of fact that determines the truth value of normative statements” and source as “causal origin of value-based thinking and language”. And there is no reason why the “source of values” should be the same for each of those readings. Though obviously they could be.
So under “conscious cognition”: traditional Kantian deontology is discovered by conscious cognition but the facts that make it true are supposed to be external to human minds (purportedly they exist in reason itself, like mathematical truths). But there are definitely also people who hold the view that what makes a decision a moral one is just that it was arrived at by the properly and rationally reflecting on values.
I don’t think any position can be accurately categorized without understanding how that position answers each of the above readings of “source”. So for instance, my own position would be that the causal origin of values is a synthesis of culturally constructed intuitions and innate, evolved intuitions on which we perform conscious cognitive operations (which are themselves a synthesis of innate abilities and technologies). So a mix between conscious and unconscious cognition, other humans, and some degree of reason. I don’t think there are any facts that make value language true and method of discovery is just what we think we’re doing with the above. It’s then helpful to ask something like: well how do you want a superintelligence to decide what to do? The answer to which somehow involves recursing the causal source of value judgments upon itself. E.g. It’s the same as asking “How ought a super intelligence behave?” which is just an instance of a value judgment. But obviously it is really difficult to a) precisely define that process and b)determine how exactly that recursion ought to happen.
You make a good point. Thanks.