I feel like there are two different concerns you’ve been expressing in your post history:
(1) Human “philosophical vulnerabilities” might get worsened (bad incentive setting, addictive technology) or exploited in the AI transition. In theory and ideally, AI could also be a solution to this and be used to make humans more philosophically robust.
(2) The importance of “solving metaphilosophy,” why doing so would help us with (1).
My view is that (1) is very important and you’re correct to highlight it as a focus area we should do more in. For some specific vulnerabilities or failure modes, I wrote a non-exhaustive list here in this post under the headings “Reflection strategies require judgment calls” and “Pitfalls of reflection procedures.” Some of it was inspired by your LW comments.
Regarding (2), I think you overestimate how difficult the problem is. My specific guess is you might overestimate its difficulty because you might confuse uncertainty over a problem with objective solutions with indecisiveness about mutually incompatible ways of reasoning. Uncertainty and indecisiveness may feel similar when you’re in that mental state, but they imply different solutions to step forward.
I feel like you already know all there is to know about metaphilosophical disagreements or solution attempts. When I read your posts, I don’t feel like “oh, I know more than Wei Dai does.” But then you seem uncertain between things that I don’t feel uncertain about, and I’m not sure what to make of that. I subscribe to the view of philosophy as “answering confused questions.” I like the following Wittgenstein’s quote:
[...] philosophers do not—or should not—supply a theory, neither do they provide explanations. “Philosophy just puts everything before us, and neither explains nor deduces anything. Since everything lies open to view there is nothing to explain (PI 126).”
As I said elsewhere, per this perspective, I see the aim of [...] philosophy as to accurately and usefully describe our option space – the different questions worth asking and how we can reason about them.
This view also works for metaphilosophical disagreements.
There’s a brand of philosophy (often associated with Oxford) that’s incompatible with the Wittgenstein quote because it uses concepts that will always remain obscure, like “objective reasons” or “objective right and wrong,” etc. The two ways of doing philosophy seem incompatible because one of them is all about concepts that the other doesn’t allow. But if you apply the perspective from the Wittgenstein quote to look at the metaphilosophical disagreement between “Wittgensteinian view” vs. “objective reasons views,” well then you’re simply choosing between two different games to play. Do you want to go down the path of increased clarity and clear questions, or do you want to go all-in on objective reasons. You gotta pick one or the other.
For what it’s worth, I feel like the prominent alignment researchers in the EA community almost exclusively reason about philosophy in the anti-realist, reductionist style. I’m reminded of Dennett’s “AI makes philosophy honest.” So, if we let alignment researchers label the training data, I’m optimistic that I’d feel satisfied with the “philosophy” we’d get out of it, conditional on solving alignment in an ambitious and comprehensive way.
Other parts of this post (the one I already linked to above) might be relevant to our disagreement, specifically with regard to the difference between uncertainty and indecisiveness.
I feel like there are two different concerns you’ve been expressing in your post history:
(1) Human “philosophical vulnerabilities” might get worsened (bad incentive setting, addictive technology) or exploited in the AI transition. In theory and ideally, AI could also be a solution to this and be used to make humans more philosophically robust.
(2) The importance of “solving metaphilosophy,” why doing so would help us with (1).
My view is that (1) is very important and you’re correct to highlight it as a focus area we should do more in. For some specific vulnerabilities or failure modes, I wrote a non-exhaustive list here in this post under the headings “Reflection strategies require judgment calls” and “Pitfalls of reflection procedures.” Some of it was inspired by your LW comments.
Regarding (2), I think you overestimate how difficult the problem is. My specific guess is you might overestimate its difficulty because you might confuse uncertainty over a problem with objective solutions with indecisiveness about mutually incompatible ways of reasoning. Uncertainty and indecisiveness may feel similar when you’re in that mental state, but they imply different solutions to step forward.
I feel like you already know all there is to know about metaphilosophical disagreements or solution attempts. When I read your posts, I don’t feel like “oh, I know more than Wei Dai does.” But then you seem uncertain between things that I don’t feel uncertain about, and I’m not sure what to make of that. I subscribe to the view of philosophy as “answering confused questions.” I like the following Wittgenstein’s quote:
As I said elsewhere, per this perspective, I see the aim of [...] philosophy as to accurately and usefully describe our option space – the different questions worth asking and how we can reason about them.
This view also works for metaphilosophical disagreements.
There’s a brand of philosophy (often associated with Oxford) that’s incompatible with the Wittgenstein quote because it uses concepts that will always remain obscure, like “objective reasons” or “objective right and wrong,” etc. The two ways of doing philosophy seem incompatible because one of them is all about concepts that the other doesn’t allow. But if you apply the perspective from the Wittgenstein quote to look at the metaphilosophical disagreement between “Wittgensteinian view” vs. “objective reasons views,” well then you’re simply choosing between two different games to play. Do you want to go down the path of increased clarity and clear questions, or do you want to go all-in on objective reasons. You gotta pick one or the other.
For what it’s worth, I feel like the prominent alignment researchers in the EA community almost exclusively reason about philosophy in the anti-realist, reductionist style. I’m reminded of Dennett’s “AI makes philosophy honest.” So, if we let alignment researchers label the training data, I’m optimistic that I’d feel satisfied with the “philosophy” we’d get out of it, conditional on solving alignment in an ambitious and comprehensive way.
Other parts of this post (the one I already linked to above) might be relevant to our disagreement, specifically with regard to the difference between uncertainty and indecisiveness.