I expect at this moment in time me building a company is going to help me deconfuse a lot of things about philosophy more than me thinking about it really hard in isolation would
Hard for me to make sense of this. What philosophical questions do you think you’ll get clarity on by doing this? What are some examples of people successfully doing this in the past?
It seems plausible that there is no such thing as “correct” metaphilosophy, and humans are just making up random stuff based on our priors and environment and that’s it and there is no “right way” to do philosophy, similar to how there are no “right preferences”.
Definitely a possibility (I’ve entertained it myself and maybe wrote some past comments along these lines). I wish there was more people studying this possibility.
I have short timelines and think we will be dead if we don’t make very rapid progress on extremely urgent practical problems like government regulation and AI safety. Metaphilosophy falls into the unfortunate bucket of “important, but not (as) urgent” in my view.
Everyone dying isn’t the worst thing that could happen. I think from a selfish perspective, I’m personally a bit more scared of surviving into a dystopia powered by ASI that is aligned in some narrow technical sense. Less sure from an altruistic/impartial perspective, but it seems at least plausible that building an aligned AI without making sure that the future human-AI civilization is “safe” is a not good thing to do.
I would say that better philosophy/arguments around questions like this is a bottleneck. One reason for my interest in metaphilosophy that I didn’t mention in the OP is that studying it seems least likely to cause harm or make things worse, compared to any other AI related topics I can work on. (I started thinking this as early as 2012.) Given how much harm people have done in the name of good, maybe we should all take “first do no harm” much more seriously?
There are no good institutions, norms, groups, funding etc to do this kind of work.
Which also represents an opportunity...
It’s weird. I happen to have a very deep interest in the topic, but it costs you weirdness points to push an idea like this when you could instead be advocating more efficiently for more pragmatic work.
Is it actually that weird? Do you have any stories of trying to talk about it with someone and having that backfire on you?
Hard for me to make sense of this. What philosophical questions do you think you’ll get clarity on by doing this? What are some examples of people successfully doing this in the past?
The fact you ask this question is interesting to me, because in my view the opposite question is the more natural one to ask: What kind of questions can you make progress on without constant grounding and dialogue with reality? This is the default of how we humans build knowledge and solve hard new questions, the places where we do best and get the least drawn astray is exactly those areas where we can have as much feedback from reality in as tight loops as possible, and so if we are trying to tackle ever more lofty problems, it becomes ever more important to get exactly that feedback wherever we can get it! From my point of view, this is the default of successful human epistemology, and the exception should be viewed with suspicion.
And for what it’s worth, acting in the real world, building a company, raising money, debating people live, building technology, making friends (and enemies), absolutely helped me become far, far less confused, and far more capable of tackling confusing problems! Actually testing my epistemology and rationality against reality, and failing (a lot), has been far more helpful for deconfusing everything from practical decision making skills to my own values than reading/thinking could have ever been in the same time span. There is value in reading and thinking, of course, but I was in a severe “thinking overhang”, and I needed to act in the world to keep learning and improving. I think most people (especially on LW) are in an “action underhang.”
“Why do people do things?” is an empirical question, it’s a thing that exists in external reality, and you need to interact with it to learn more about it. And if you want to tackle even higher level problems, you need to have even more refined feedback. When a physicist wants to understand the fundamentals of reality, they need to set up insane crazy particle accelerators and space telescopes and supercomputers and what not to squeeze bits of evidence out of reality and actually ground whatever theoretical musings they may have been thinking about. So if you want to understand the fundamentals of philosophy and the human condition, by default I expect you are going to need to do the equivalent kind of “squeezing bits out of reality”, by doing hard things such as creating institutions, building novel technology, persuading people, etc. “Building a company” is just one common example of a task that forces you to interact a lot with reality to be good.
Fundamentally, I believe that good philosophy should make you stronger and allow you to make the world better, otherwise, why are you bothering? If you actually “solve metaphilosophy”, I think the way this should end up looking is that you can now do crazy things. You can figure out new forms of science crazy fast, you can persuade billionaires to support you, you can build monumental organizations that last for generations. Or, in reverse, I expect that if you develop methods to do such impressive feats, you will necessarily have to learn deep truths about reality and the human condition, and acquire the skills you will need to tackle a task as heroic as “solving metaphilosophy.”
Everyone dying isn’t the worst thing that could happen. I think from a selfish perspective, I’m personally a bit more scared of surviving into a dystopia powered by ASI that is aligned in some narrow technical sense. Less sure from an altruistic/impartial perspective, but it seems at least plausible that building an aligned AI without making sure that the future human-AI civilization is “safe” is a not good thing to do.
I think this grounds out into object level disagreements about how we expect the future to go, probably. I think s-risks are extremely unlikely at the moment, and when I look at how best to avoid them, most such timelines don’t go through “figure out something like metaphilosophy”, but more likely through “just apply bog standard decent humanist deontological values and it’s good enough.” A lot of the s-risk in my view comes from the penchant for maximizing “good” that utilitarianism tends to promote, if we instead aim for “good enough” (which is what most people tend to instinctively favor), that cuts off most of the s-risk (though not all).
To get to the really good timelines, that route through “solve metaphilosophy”, there are mandatory previous nodes such as “don’t go extinct in 5 years.” Buying ourselves more time is powerful optionality, not just for concrete technical work, but also for improving philosophy, human epistemology/rationality, etc.
I don’t think I see a short path to communicating the parts of my model that would be most persuasive to you here (if you’re up for a call or irl discussion sometime lmk), but in short I think of policy, coordination, civilizational epistemology, institution building and metaphilosophy as closely linked and tractable problems, if only it wasn’t the case that there was a small handful of AI labs (largely supported/initiated by EA/LW-types) that are deadset on burning the commons as fast as humanly possible. If we had a few more years/decades, I think we could actually make tangible and compounding progress on these problems.
I would say that better philosophy/arguments around questions like this is a bottleneck. One reason for my interest in metaphilosophy that I didn’t mention in the OP is that studying it seems least likely to cause harm or make things worse, compared to any other AI related topics I can work on. (I started thinking this as early as 2012.) Given how much harm people have done in the name of good, maybe we should all take “first do no harm” much more seriously?
I actually respect this reasoning. I disagree strategically, but I think this is a very morally defensible position to hold, unlike the mental acrobatics necessary to work at the x-risk factories because you want to be “in the room”.
Which also represents an opportunity...
It does! If I was you, and I wanted to push forward work like this, the first thing I would do is build a company/institution! It will both test your mettle against reality and allow you to build a compounding force.
Is it actually that weird? Do you have any stories of trying to talk about it with someone and having that backfire on you?
Yup, absolutely. If you take even a microstep outside of the EA/rat-sphere, these kind of topics quickly become utterly alien to anyone. Try explaining to a politician worried about job loss, or a middle aged housewife worried about her future pension, or a young high school dropout unable to afford housing, that actually we should be worried about whether we are doing metaphilosophy correctly to ensure that future immortal superintelligence reason correctly about acausal alien gods from math-space so they don’t cause them to torture trillions of simulated souls! This is exaggerated for comedic effect, but this is really what even relatively intro level LW philosophy by default often sounds like to many people!
As the saying goes, “Grub first, then ethics.” (though I would go further and say that people’s instinctive rejection of what I would less charitably call “galaxy brain thinking” is actually often well calibrated)
You raised a very interesting point in the last comment, that metaphilosophy already encompasses everything, that we could conceive of at least.
So a ‘solution’ is not tractable due to various well known issues such as the halting problem and so on. (Though perhaps in the very distant future this could be different.)
However this leads to a problem, as exemplified by your phrasing here:
Fundamentally, I believe that good philosophy should make you stronger and allow you to make the world better, otherwise, why are you bothering …
‘good philosophy’ is not a sensible category since you already know you have not, and cannot, ‘solve’ metaphilosophy. Nor can any other LW reader do so.
‘good’ or ‘bad’ in real practice are, at best, whatever the popular consensus is in the present reality, at worst, just someone’s idiosyncratic opinions.
Very few concepts are entirely independent from any philosophical or metaphilosophical implications whatsoever, and ‘good philosophy’ is not one of them.
But you still felt a need to attach these modifiers, due to a variety of reasons well analyzed on LW, so the pretenseof a solved or solvable metaphilosophy is still needed for this part of the comment to make sense.
I don’t want to single out your comment too much though, since it’s just the most convenient example, this applies to most LW comments.
i.e. If everyone actually accepted the point, which I agree with, I dare say a huge chunk of LW comments are close to meaningless from a formal viewpoint, or at least very open to interpretation by anyone who isn’t immersed in 21st century human culture.
“good” always refers to idiosyncratic opinions, I don’t really take moral realism particularly seriously. I think there is “good” philosophy in the same way there are “good” optimization algorithms for neural networks, while also I assume there is no one optimizer that “solves” all neural network problems.
‘”good” optimization algorithms for neural networks’ also has no difference in meaning from ‘”glorxnag” optimization algorithms for neural networks’, or any random permutation, if your prior point holds.
I don’t understand what point you are trying to make, to be honest. There are certain problems that humans/I care about that we/I want NNs to solve, and some optimizers (e.g. Adam) solve those problems better or more tractably than others (e.g. SGD or second order methods). You can claim that the “set of problems humans care about” is “arbitrary”, to which I would reply “sure?”
Similarly, I want “good” “philosophy” to be “better” at “solving” “problems I care about.” If you want to use other words for this, my answer is again “sure?” I think this is a good use of the word “philosophy” that gets better at what people actually want out of it, but I’m not gonna die on this hill because of an abstract semantic disagreement.
That’s the thing, there is no definable “set of problems humans care about” without some kind of attached or presumed metaphilosophy,at least none that you, or anyone, could possibly figure out in the foreseeable future and prove to a reasonable degree of confidence to the LW readerbase.
It’s not even ‘arbitrary’, that string of letters is indistinguishable from random noise.
i.e. Right now your first paragraph is mostly meaningless if read completely literally and by someone who accepts the claim. Such a hypothetical person would think you’ve gone nuts because it would appear like you took a well written comment and inserted strings of random keyboard bashing in the middle.
Of course it’s unlikely that someone would be so literal minded, and so insistent on logical correctness, that they would completely equate it with random bashing of a keyboard. But it’s possible some portion of readers lean towards that.
Hard for me to make sense of this. What philosophical questions do you think you’ll get clarity on by doing this? What are some examples of people successfully doing this in the past?
Definitely a possibility (I’ve entertained it myself and maybe wrote some past comments along these lines). I wish there was more people studying this possibility.
Everyone dying isn’t the worst thing that could happen. I think from a selfish perspective, I’m personally a bit more scared of surviving into a dystopia powered by ASI that is aligned in some narrow technical sense. Less sure from an altruistic/impartial perspective, but it seems at least plausible that building an aligned AI without making sure that the future human-AI civilization is “safe” is a not good thing to do.
I would say that better philosophy/arguments around questions like this is a bottleneck. One reason for my interest in metaphilosophy that I didn’t mention in the OP is that studying it seems least likely to cause harm or make things worse, compared to any other AI related topics I can work on. (I started thinking this as early as 2012.) Given how much harm people have done in the name of good, maybe we should all take “first do no harm” much more seriously?
Which also represents an opportunity...
Is it actually that weird? Do you have any stories of trying to talk about it with someone and having that backfire on you?
The fact you ask this question is interesting to me, because in my view the opposite question is the more natural one to ask: What kind of questions can you make progress on without constant grounding and dialogue with reality? This is the default of how we humans build knowledge and solve hard new questions, the places where we do best and get the least drawn astray is exactly those areas where we can have as much feedback from reality in as tight loops as possible, and so if we are trying to tackle ever more lofty problems, it becomes ever more important to get exactly that feedback wherever we can get it! From my point of view, this is the default of successful human epistemology, and the exception should be viewed with suspicion.
And for what it’s worth, acting in the real world, building a company, raising money, debating people live, building technology, making friends (and enemies), absolutely helped me become far, far less confused, and far more capable of tackling confusing problems! Actually testing my epistemology and rationality against reality, and failing (a lot), has been far more helpful for deconfusing everything from practical decision making skills to my own values than reading/thinking could have ever been in the same time span. There is value in reading and thinking, of course, but I was in a severe “thinking overhang”, and I needed to act in the world to keep learning and improving. I think most people (especially on LW) are in an “action underhang.”
“Why do people do things?” is an empirical question, it’s a thing that exists in external reality, and you need to interact with it to learn more about it. And if you want to tackle even higher level problems, you need to have even more refined feedback. When a physicist wants to understand the fundamentals of reality, they need to set up insane crazy particle accelerators and space telescopes and supercomputers and what not to squeeze bits of evidence out of reality and actually ground whatever theoretical musings they may have been thinking about. So if you want to understand the fundamentals of philosophy and the human condition, by default I expect you are going to need to do the equivalent kind of “squeezing bits out of reality”, by doing hard things such as creating institutions, building novel technology, persuading people, etc. “Building a company” is just one common example of a task that forces you to interact a lot with reality to be good.
Fundamentally, I believe that good philosophy should make you stronger and allow you to make the world better, otherwise, why are you bothering? If you actually “solve metaphilosophy”, I think the way this should end up looking is that you can now do crazy things. You can figure out new forms of science crazy fast, you can persuade billionaires to support you, you can build monumental organizations that last for generations. Or, in reverse, I expect that if you develop methods to do such impressive feats, you will necessarily have to learn deep truths about reality and the human condition, and acquire the skills you will need to tackle a task as heroic as “solving metaphilosophy.”
I think this grounds out into object level disagreements about how we expect the future to go, probably. I think s-risks are extremely unlikely at the moment, and when I look at how best to avoid them, most such timelines don’t go through “figure out something like metaphilosophy”, but more likely through “just apply bog standard decent humanist deontological values and it’s good enough.” A lot of the s-risk in my view comes from the penchant for maximizing “good” that utilitarianism tends to promote, if we instead aim for “good enough” (which is what most people tend to instinctively favor), that cuts off most of the s-risk (though not all).
To get to the really good timelines, that route through “solve metaphilosophy”, there are mandatory previous nodes such as “don’t go extinct in 5 years.” Buying ourselves more time is powerful optionality, not just for concrete technical work, but also for improving philosophy, human epistemology/rationality, etc.
I don’t think I see a short path to communicating the parts of my model that would be most persuasive to you here (if you’re up for a call or irl discussion sometime lmk), but in short I think of policy, coordination, civilizational epistemology, institution building and metaphilosophy as closely linked and tractable problems, if only it wasn’t the case that there was a small handful of AI labs (largely supported/initiated by EA/LW-types) that are deadset on burning the commons as fast as humanly possible. If we had a few more years/decades, I think we could actually make tangible and compounding progress on these problems.
I actually respect this reasoning. I disagree strategically, but I think this is a very morally defensible position to hold, unlike the mental acrobatics necessary to work at the x-risk factories because you want to be “in the room”.
It does! If I was you, and I wanted to push forward work like this, the first thing I would do is build a company/institution! It will both test your mettle against reality and allow you to build a compounding force.
Yup, absolutely. If you take even a microstep outside of the EA/rat-sphere, these kind of topics quickly become utterly alien to anyone. Try explaining to a politician worried about job loss, or a middle aged housewife worried about her future pension, or a young high school dropout unable to afford housing, that actually we should be worried about whether we are doing metaphilosophy correctly to ensure that future immortal superintelligence reason correctly about acausal alien gods from math-space so they don’t cause them to torture trillions of simulated souls! This is exaggerated for comedic effect, but this is really what even relatively intro level LW philosophy by default often sounds like to many people!
As the saying goes, “Grub first, then ethics.” (though I would go further and say that people’s instinctive rejection of what I would less charitably call “galaxy brain thinking” is actually often well calibrated)
You raised a very interesting point in the last comment, that metaphilosophy already encompasses everything, that we could conceive of at least.
So a ‘solution’ is not tractable due to various well known issues such as the halting problem and so on. (Though perhaps in the very distant future this could be different.)
However this leads to a problem, as exemplified by your phrasing here:
‘good philosophy’ is not a sensible category since you already know you have not, and cannot, ‘solve’ metaphilosophy. Nor can any other LW reader do so.
‘good’ or ‘bad’ in real practice are, at best, whatever the popular consensus is in the present reality, at worst, just someone’s idiosyncratic opinions.
Very few concepts are entirely independent from any philosophical or metaphilosophical implications whatsoever, and ‘good philosophy’ is not one of them.
But you still felt a need to attach these modifiers, due to a variety of reasons well analyzed on LW, so the pretense of a solved or solvable metaphilosophy is still needed for this part of the comment to make sense.
I don’t want to single out your comment too much though, since it’s just the most convenient example, this applies to most LW comments.
i.e. If everyone actually accepted the point, which I agree with, I dare say a huge chunk of LW comments are close to meaningless from a formal viewpoint, or at least very open to interpretation by anyone who isn’t immersed in 21st century human culture.
“good” always refers to idiosyncratic opinions, I don’t really take moral realism particularly seriously. I think there is “good” philosophy in the same way there are “good” optimization algorithms for neural networks, while also I assume there is no one optimizer that “solves” all neural network problems.
‘”good” optimization algorithms for neural networks’ also has no difference in meaning from ‘”glorxnag” optimization algorithms for neural networks’, or any random permutation, if your prior point holds.
I don’t understand what point you are trying to make, to be honest. There are certain problems that humans/I care about that we/I want NNs to solve, and some optimizers (e.g. Adam) solve those problems better or more tractably than others (e.g. SGD or second order methods). You can claim that the “set of problems humans care about” is “arbitrary”, to which I would reply “sure?”
Similarly, I want “good” “philosophy” to be “better” at “solving” “problems I care about.” If you want to use other words for this, my answer is again “sure?” I think this is a good use of the word “philosophy” that gets better at what people actually want out of it, but I’m not gonna die on this hill because of an abstract semantic disagreement.
That’s the thing, there is no definable “set of problems humans care about” without some kind of attached or presumed metaphilosophy, at least none that you, or anyone, could possibly figure out in the foreseeable future and prove to a reasonable degree of confidence to the LW readerbase.
It’s not even ‘arbitrary’, that string of letters is indistinguishable from random noise.
i.e. Right now your first paragraph is mostly meaningless if read completely literally and by someone who accepts the claim. Such a hypothetical person would think you’ve gone nuts because it would appear like you took a well written comment and inserted strings of random keyboard bashing in the middle.
Of course it’s unlikely that someone would be so literal minded, and so insistent on logical correctness, that they would completely equate it with random bashing of a keyboard. But it’s possible some portion of readers lean towards that.
That is not a fact.
Hear! Hear!