I’d suggest that trying to understand what values are would potentially have been a better direction to emphasize. Our understanding here is still pre-Socratic, basically pre-cultural.
It seems to me that values have been a main focus of philosophy for a long time, with moral philosophy (or perhaps meta-ethics if the topic is “what values are”) devoted to it and discussed frequently both in academia and out, whereas metaphilosophy has received much less attention. This implies that we know progress on understanding values is probably pretty hard on the current margins, whereas there’s a lot more uncertainty about the difficulty of metaphilosophy. Solving the latter would also be of greater utility, since it makes solving all other philosophical problems easier, not just values. I’m curious about the rationale behind your suggestion.
It seems to me that values have been a main focus of philosophy for a long time
I’m curious about the rationale behind your suggestion.
Specifically the question of “what values are” I don’t think has been addressed (I’ve looked around some, but certainly not thoroughly). A key problem with previous philosophy is that values are extreme in how much they require some sort of mental context (https://www.lesswrong.com/posts/HJ4EHPG5qPbbbk5nK/gemini-modeling). Previous philosophy (that I’m aware of) largely takes the mental context for granted, or only highlights the parts of it that are called into question, or briefly touches on it. This is pretty reasonable if you’re a human talking to humans, because you do probably share most of that mental context. But it fails on two sorts of cases:
trying to think about or grow/construct/shape really alien minds, like AIs;
trying to exert human values in a way that is good but unnatural (think for example of governments, teams, “superhuman devotion to a personal mission”, etc.).
The latter, 2., might have, given more progress, helped us be wiser.
My comment was responding to
it was a bad idea to invent things like logic, mathematical proofs, and scientific methodologies, because it permanently accelerated the wrong things (scientific and technological progress) while giving philosophy only a temporary boost (by empowering the groups that invented those things, which had better than average philosophical competence, to spread their culture/influence).
So I’m saying, in retrospect on the 2.5 millennia of philosophy, it plausibly would have been better to have an “organology, physiology, medicine, and medical enhancement” of values. To say it a different way, we should have been building the conceptual and introspective foundations that would have provided the tools with which we might have been able to become much wiser than is accessible to the lone investigators who periodically arise, try to hack their way a small ways up the mountain, and then die, leaving mostly only superficial transmissions.
whereas metaphilosophy has received much less attention.
I would agree pretty strongly with some version of “metaphilosophy is potentially a very underserved investment opportunity”, though we don’t necessarily agree (because of having “very different tastes” about what metaphilosophy should be, amounting to not even talking about the same thing). I have ranted several times to friends about how philosophy (by which I mean metaphilosophy—under one description, something like “recursive communal yak-shaving aimed at the (human-)canonical”) has barely ever been tried, etc.
I know @Wei Dai’s post isn’t entirely serious, but I want to flag that the position that we could have understood values/philosophy without knowing about math/logic is a fictional world/fabricated option.
It cannot exist, and updateleness can be taken too far with compute constraints.
I want to flag that the position that we could have understood values/philosophy without knowing about math/logic is a fictional world/fabricated option.
Maybe but I don’t believe that you know this. Lots of important concepts want to be gotten at by routes that don’t use much math or use quite different math from “math to understand computers” or “math to formalize epistemology”. Darwin didn’t need much math to get lots of the core structure of evolution by natural selection on random mutation.
Perhaps more seriously, the philosophers who got a temporary manpower and influence boost from the invention of math and science should have worked much harder to solve metaphilosophy, while they had the advantage.
This contradicts my position in Some Thoughts on Metaphilosophy. What about that post do you find unconvincing, or what is your own argument for “philosophy being insoluble”?
Talk about “values” is very popular on LessWrong, but much less common in philosophy or natural language. I confess I don’t even know what you mean with “trying to understand what values are”. Can you make the problem statement more precise, perhaps without reference to “values”?
Old High rationalism had a world view in which rationality, AI and ethics were all roughly the same thing: the optimisation of utility/value (which are the same thing). Rationality was von Neumann rationality; an AI was a utility maximising agent; the only possible theory of ethics is utilitarianism, IE utility maximisation, IE value maximisation.
Philosophers have discussed these under the term “desires”. I think there was a lot of progress since the time of the pre-Socratics. Aristotle’s practical syllogism, Buridan’s donkey, Hume emphasis of the independence of beliefs and desires, Kant’s distinction between theoretical reason and practical reason, direction of fit, Richard Jeffrey’s utility theory (where utilities are degrees of desire), analysis of akrasia by various analytic philosophers, Nozick’s experience machine, and various others.
“A lot of progress”.… well, reality doesn’t grade on a curve. Surely someone has said something about something, yes, but have we said enough about what matters? Not even close. If you don’t know how inadequate our understanding of values is I can’t convince you in a comment, but one way to find out would be to try to solve alignment. E.g. see https://tsvibt.blogspot.com/2023/03/the-fraught-voyage-of-aligned-novelty.html
There is quite the difference between “our understanding is still pre-Socratic” and “we haven’t said enough”. In general I think very few people here (not sure whether this applies to you) are familiar with the philosophical literature on topics in this area. For example, there is very little interest on LessWrong in normative ethics and the associated philosophical research. Even though this is directly related to alignment, since, if you you have an intent-aligned ASI (which is probably easier to achieve than shooting straight for value alignment) you probably need to know what ethics it should implement when asking it to create a fully value-aligned ASI.
Interestingly, the situation is quite different for the EA Forum, where there are regular high-quality posts on solving issues in normative ethics with reference to the academic literature, like the repugnant conclusion, the procreation asymmetry and the status of person-affecting theories. Any satisfactory normative ethical theory needs to solve these problems, similar to how any satisfactory normative theory of epistemic rationality needs to solve the various epistemic paradoxes and related issues.
Again, I don’t know whether this applies to you, but most cases of “philosophy has made basically no progress on topic X” seem to come from people who have very little knowledge of the philosophical literature on topic X.
I’m not sure. I did put in some effort to survey various strands of philosophy related to axiology, but not much effort. E.g. looked at some writings in the vein of Anscombe’s study of intention; tried to read D+G because maybe “machines” is the sort of thing I’m asking about (was not useful to me lol); have read some Heidegger; some Nietzsche; some more obscure things like “Care Crosses the River” by Blumenberg; the basics of the “analytical” stuff LWers know (including doing some of my own research on decision theory); etc etc. But in short, no, none of it even addresses the question—and the failure is the sort of failure that was supposed to have its coarsest outlines brought to light by genuinely Socratic questioning, which is why I call it “pre-Socratic”, not to say that “no one since Socrates has billed themselves as talking about something related to values or something”.
I think even communicating the question would take a lot of work, which as I said is part of the problem. A couple hints:
You should think of the question of values as being more like “what is the driving engine” rather than “what are the rules” or “what are the outcomes” or “how to make decisions” etc.
I’d suggest that trying to understand what values are would potentially have been a better direction to emphasize. Our understanding here is still pre-Socratic, basically pre-cultural.
It seems to me that values have been a main focus of philosophy for a long time, with moral philosophy (or perhaps meta-ethics if the topic is “what values are”) devoted to it and discussed frequently both in academia and out, whereas metaphilosophy has received much less attention. This implies that we know progress on understanding values is probably pretty hard on the current margins, whereas there’s a lot more uncertainty about the difficulty of metaphilosophy. Solving the latter would also be of greater utility, since it makes solving all other philosophical problems easier, not just values. I’m curious about the rationale behind your suggestion.
Specifically the question of “what values are” I don’t think has been addressed (I’ve looked around some, but certainly not thoroughly). A key problem with previous philosophy is that values are extreme in how much they require some sort of mental context (https://www.lesswrong.com/posts/HJ4EHPG5qPbbbk5nK/gemini-modeling). Previous philosophy (that I’m aware of) largely takes the mental context for granted, or only highlights the parts of it that are called into question, or briefly touches on it. This is pretty reasonable if you’re a human talking to humans, because you do probably share most of that mental context. But it fails on two sorts of cases:
trying to think about or grow/construct/shape really alien minds, like AIs;
trying to exert human values in a way that is good but unnatural (think for example of governments, teams, “superhuman devotion to a personal mission”, etc.).
The latter, 2., might have, given more progress, helped us be wiser.
My comment was responding to
So I’m saying, in retrospect on the 2.5 millennia of philosophy, it plausibly would have been better to have an “organology, physiology, medicine, and medical enhancement” of values. To say it a different way, we should have been building the conceptual and introspective foundations that would have provided the tools with which we might have been able to become much wiser than is accessible to the lone investigators who periodically arise, try to hack their way a small ways up the mountain, and then die, leaving mostly only superficial transmissions.
I would agree pretty strongly with some version of “metaphilosophy is potentially a very underserved investment opportunity”, though we don’t necessarily agree (because of having “very different tastes” about what metaphilosophy should be, amounting to not even talking about the same thing). I have ranted several times to friends about how philosophy (by which I mean metaphilosophy—under one description, something like “recursive communal yak-shaving aimed at the (human-)canonical”) has barely ever been tried, etc.
I know @Wei Dai’s post isn’t entirely serious, but I want to flag that the position that we could have understood values/philosophy without knowing about math/logic is a fictional world/fabricated option.
It cannot exist, and updateleness can be taken too far with compute constraints.
Maybe but I don’t believe that you know this. Lots of important concepts want to be gotten at by routes that don’t use much math or use quite different math from “math to understand computers” or “math to formalize epistemology”. Darwin didn’t need much math to get lots of the core structure of evolution by natural selection on random mutation.
Perhaps more seriously, the philosophers who got a temporary manpower and influence boost from the invention of math and science should have worked much harder to solve metaphilosophy, while they had the advantage.
It’s quite possible that we have solved metaphilosophy, in the direction of philosophy being insoluble.
This contradicts my position in Some Thoughts on Metaphilosophy. What about that post do you find unconvincing, or what is your own argument for “philosophy being insoluble”?
Talk about “values” is very popular on LessWrong, but much less common in philosophy or natural language. I confess I don’t even know what you mean with “trying to understand what values are”. Can you make the problem statement more precise, perhaps without reference to “values”?
Old High rationalism had a world view in which rationality, AI and ethics were all roughly the same thing: the optimisation of utility/value (which are the same thing). Rationality was von Neumann rationality; an AI was a utility maximising agent; the only possible theory of ethics is utilitarianism, IE utility maximisation, IE value maximisation.
No, that’s part of the problem. There’s pretheoretic material as some of a starting point here:
https://www.lesswrong.com/posts/YLRPhvgN4uZ6LCLxw/human-wanting
Whatever those things are, you’d want to understand the context that makes them what they are:
https://www.lesswrong.com/posts/HJ4EHPG5qPbbbk5nK/gemini-modeling
And refactor the big blob into lots of better concepts, which would probably require a larger investigation and conceptual refactoring:
https://www.lesswrong.com/posts/TNQKFoWhAkLCB4Kt7/a-hermeneutic-net-for-agency
In particular so that we understand how “values” can be stable (https://www.lesswrong.com/posts/Ht4JZtxngKwuQ7cDC/tsvibt-s-shortform?commentId=koeti9ygXB9wPLnnF) and can incorporate novel concepts / deal with novel domains (https://www.lesswrong.com/posts/CBHpzpzJy98idiSGs/do-humans-derive-values-from-fictitious-imputed-coherence) and eventually address the stuff here: https://www.lesswrong.com/posts/ASZco85chGouu2LKk/the-fraught-voyage-of-aligned-novelty
Philosophers have discussed these under the term “desires”. I think there was a lot of progress since the time of the pre-Socratics. Aristotle’s practical syllogism, Buridan’s donkey, Hume emphasis of the independence of beliefs and desires, Kant’s distinction between theoretical reason and practical reason, direction of fit, Richard Jeffrey’s utility theory (where utilities are degrees of desire), analysis of akrasia by various analytic philosophers, Nozick’s experience machine, and various others.
“A lot of progress”.… well, reality doesn’t grade on a curve. Surely someone has said something about something, yes, but have we said enough about what matters? Not even close. If you don’t know how inadequate our understanding of values is I can’t convince you in a comment, but one way to find out would be to try to solve alignment. E.g. see https://tsvibt.blogspot.com/2023/03/the-fraught-voyage-of-aligned-novelty.html
There is quite the difference between “our understanding is still pre-Socratic” and “we haven’t said enough”. In general I think very few people here (not sure whether this applies to you) are familiar with the philosophical literature on topics in this area. For example, there is very little interest on LessWrong in normative ethics and the associated philosophical research. Even though this is directly related to alignment, since, if you you have an intent-aligned ASI (which is probably easier to achieve than shooting straight for value alignment) you probably need to know what ethics it should implement when asking it to create a fully value-aligned ASI.
Interestingly, the situation is quite different for the EA Forum, where there are regular high-quality posts on solving issues in normative ethics with reference to the academic literature, like the repugnant conclusion, the procreation asymmetry and the status of person-affecting theories. Any satisfactory normative ethical theory needs to solve these problems, similar to how any satisfactory normative theory of epistemic rationality needs to solve the various epistemic paradoxes and related issues.
Again, I don’t know whether this applies to you, but most cases of “philosophy has made basically no progress on topic X” seem to come from people who have very little knowledge of the philosophical literature on topic X.
I’m not sure. I did put in some effort to survey various strands of philosophy related to axiology, but not much effort. E.g. looked at some writings in the vein of Anscombe’s study of intention; tried to read D+G because maybe “machines” is the sort of thing I’m asking about (was not useful to me lol); have read some Heidegger; some Nietzsche; some more obscure things like “Care Crosses the River” by Blumenberg; the basics of the “analytical” stuff LWers know (including doing some of my own research on decision theory); etc etc. But in short, no, none of it even addresses the question—and the failure is the sort of failure that was supposed to have its coarsest outlines brought to light by genuinely Socratic questioning, which is why I call it “pre-Socratic”, not to say that “no one since Socrates has billed themselves as talking about something related to values or something”.
I think even communicating the question would take a lot of work, which as I said is part of the problem. A couple hints:
https://www.lesswrong.com/posts/NqsNYsyoA2YSbb3py/fundamental-question-what-determines-a-mind-s-effects (I think if you read this it will seem incredibly boringly obvious and trivial, and yet, literally no one addresses it! Some people sort of try, but fail so badly that it can’t count as progress. Closest would be some bits of theology, maybe? Not sure.)
https://www.lesswrong.com/posts/p7mMJvwDbuvo4K7NE/telopheme-telophore-and-telotect (I think this distinction is mostly a failed attempt to carve things, but the question that it fails to answer is related to the important question of values.)
You should think of the question of values as being more like “what is the driving engine” rather than “what are the rules” or “what are the outcomes” or “how to make decisions” etc.