All else equal, I prefer an AI which is not capable to philosophy, as I am afraid of completely alien conclusions which it could come to (e.g. insect are more important than humans).
More over, I am skeptical that going on meta-level simplifies the problem to the level that it will be solvable by humans (the same about meta-ethics and theory of human values). For example, if someone says that he is not able to understand math, but instead will work on meta-mathematical problems, we would be skeptical about his ability to contribute. Why meta-level would be simpler?
More over, I am skeptical that going on meta-level simplifies the problem to the level that it will be solvable by humans (the same about meta-ethics and theory of human values).
This is also my reason for being pessimistic about solving metaphilosophy before a good number of object-level philosophical problems have been solved (e.g. in decision theory, ontology/metaphysics, and epistemology). If we imagine being in a state where we believe running computation X would solve hard philosophical problem Y, then it would seem that we already have a great deal of philosophical knowledge about Y, or a more general class of problems that includes Y.
More generally, we could look at the history difficulty of solving a problem vs. the difficulty of automating it. For example: the difficulty of walking vs. the difficulty of programming a robot to walk; the difficulty of adding numbers vs. the difficulty of specifying an addition algorithm; the difficulty of discovering electricity vs. the difficulty of solving philosophy of science to the point where it’s clear how a reasoner could have discovered (and been confident in) electricity; and so on.
The plausible story I have that looks most optimistic for metaphilosophy looks something like:
Some philosophical community makes large progress on a bunch of philosophical problems, at a high level of technical sophistication.
As part of their work, they discover some “generators” that generate a bunch of the object-level solutions when translated across domains; these generators might involve e.g. translating a philosophical problem to one of a number of standard forms and then solving the standard form.
They also find philosophical reasons to believe that these generators will generate good object-level solutions to new problems, not just the ones that have already been studied.
These generators would then constitute a solution to metaphilosophy.
I think our positions on this are pretty close, but I may put a bit more weight on other “plausible stories” for solving metaphilosophy relative to your “plausible story”. (I’m not sure if overall I’m more or less optimistic than you are.)
If we imagine being in a state where we believe running computation X would solve hard philosophical problem Y, then it would seem that we already have a great deal of philosophical knowledge about Y, or a more general class of problems that includes Y.
It seems quite possible that understanding the general class of problems that includes Y is easier than understanding Y itself, and that allows us to find a computation X that would solve Y without much understanding of Y itself. As an analogy, suppose Y is some complex decision problem that we have little understanding of, and X is an AI that is programmed with a good decision theory.
More generally, we could look at the history difficulty of solving a problem vs. the difficulty of automating it. For example: the difficulty of walking vs. the difficulty of programming a robot to walk;
This does not seem like a very strong argument for your position. My suggestion in the OP is that humans already know the equivalent of “walking” (i.e., doing philosophy), we’re just doing it very slowly. Given this, your analogies don’t seem very conclusive about the difficulty of solving metaphilosophy or whether we have to make a bunch more progress on object-level philosophical problems before we can solve metaphilosophy.
Creating AI for solving hard philosophical problems is like passing hot potato from right hand to left.
For example, I want to solve the problem of qualia. I can’t solve it myself, but may be I can create super-intelligent AI which will help me to solve it? Now I start to working on AI, and soon encounter the the control problem. Trying to solve the control problem, I would have to specify nature of human values, and soon I will find the need to tell something about existing and nature of qualia. Now the circle is done: I have the same problem of qualia, but packed inside the control problem. If I make some assumption about what qualia should be, they will probably affect the final answer by AI.
However, I still could use some forms of AI to solve qualia problem: if I use google search, I could quickly find all relevant articles, identify the most cited, newest, maybe create an argument map. This is where Drexler’s CAIS may help.
Maybe one AI philosophy service could look like: would ask you a bunch of other questions that are simpler than the problem of qualia, then show you what those answers imply about the problem of qualia if you use some method of reconciling those answers.
In fact, when I use Google Scholar to find new articles about e.g. qualia, I already use narrow AI to advance my understanding. So AI could be useful in thinking about philosophical problems. What I am afraid of is AI’s decisions based on incomprehensible AI-created philosophy.
More over, I am skeptical that going on meta-level simplifies the problem to the level that it will be solvable by humans
If I gave the impression in this post that I expect metaphilosophy to be solved before someone builds an AGI, that was far from my intentions. I think this is a small-chance-of-high-return kind of situation, plus I think someone has to try to attack the problem if only to generate evidence that it really is a hard problem, otherwise I don’t know how to convince people to adopt costly social solutions like stopping technological progress. (And actually I don’t expect the evidence to be highly persuasive either, so this amounts to just another small chance of high return.)
What I wrote in an earlier post still describes my overall position:
There is no strong empirical evidence that solving metaphilosophy is superhumanly difficult, simply because not many people have attempted to solve it. But I don’t think that a reasonable prior combined with what evidence we do have (i.e., absence of visible progress or clear hints as to how to proceed) gives much hope for optimism either.
All else equal, I prefer an AI which is not capable to philosophy, as I am afraid of completely alien conclusions which it could come to (e.g. insect are more important than humans).
More over, I am skeptical that going on meta-level simplifies the problem to the level that it will be solvable by humans (the same about meta-ethics and theory of human values). For example, if someone says that he is not able to understand math, but instead will work on meta-mathematical problems, we would be skeptical about his ability to contribute. Why meta-level would be simpler?
This is also my reason for being pessimistic about solving metaphilosophy before a good number of object-level philosophical problems have been solved (e.g. in decision theory, ontology/metaphysics, and epistemology). If we imagine being in a state where we believe running computation X would solve hard philosophical problem Y, then it would seem that we already have a great deal of philosophical knowledge about Y, or a more general class of problems that includes Y.
More generally, we could look at the history difficulty of solving a problem vs. the difficulty of automating it. For example: the difficulty of walking vs. the difficulty of programming a robot to walk; the difficulty of adding numbers vs. the difficulty of specifying an addition algorithm; the difficulty of discovering electricity vs. the difficulty of solving philosophy of science to the point where it’s clear how a reasoner could have discovered (and been confident in) electricity; and so on.
The plausible story I have that looks most optimistic for metaphilosophy looks something like:
Some philosophical community makes large progress on a bunch of philosophical problems, at a high level of technical sophistication.
As part of their work, they discover some “generators” that generate a bunch of the object-level solutions when translated across domains; these generators might involve e.g. translating a philosophical problem to one of a number of standard forms and then solving the standard form.
They also find philosophical reasons to believe that these generators will generate good object-level solutions to new problems, not just the ones that have already been studied.
These generators would then constitute a solution to metaphilosophy.
I think our positions on this are pretty close, but I may put a bit more weight on other “plausible stories” for solving metaphilosophy relative to your “plausible story”. (I’m not sure if overall I’m more or less optimistic than you are.)
It seems quite possible that understanding the general class of problems that includes Y is easier than understanding Y itself, and that allows us to find a computation X that would solve Y without much understanding of Y itself. As an analogy, suppose Y is some complex decision problem that we have little understanding of, and X is an AI that is programmed with a good decision theory.
This does not seem like a very strong argument for your position. My suggestion in the OP is that humans already know the equivalent of “walking” (i.e., doing philosophy), we’re just doing it very slowly. Given this, your analogies don’t seem very conclusive about the difficulty of solving metaphilosophy or whether we have to make a bunch more progress on object-level philosophical problems before we can solve metaphilosophy.
Creating AI for solving hard philosophical problems is like passing hot potato from right hand to left.
For example, I want to solve the problem of qualia. I can’t solve it myself, but may be I can create super-intelligent AI which will help me to solve it? Now I start to working on AI, and soon encounter the the control problem. Trying to solve the control problem, I would have to specify nature of human values, and soon I will find the need to tell something about existing and nature of qualia. Now the circle is done: I have the same problem of qualia, but packed inside the control problem. If I make some assumption about what qualia should be, they will probably affect the final answer by AI.
However, I still could use some forms of AI to solve qualia problem: if I use google search, I could quickly find all relevant articles, identify the most cited, newest, maybe create an argument map. This is where Drexler’s CAIS may help.
Maybe one AI philosophy service could look like: would ask you a bunch of other questions that are simpler than the problem of qualia, then show you what those answers imply about the problem of qualia if you use some method of reconciling those answers.
In fact, when I use Google Scholar to find new articles about e.g. qualia, I already use narrow AI to advance my understanding. So AI could be useful in thinking about philosophical problems. What I am afraid of is AI’s decisions based on incomprehensible AI-created philosophy.
If I gave the impression in this post that I expect metaphilosophy to be solved before someone builds an AGI, that was far from my intentions. I think this is a small-chance-of-high-return kind of situation, plus I think someone has to try to attack the problem if only to generate evidence that it really is a hard problem, otherwise I don’t know how to convince people to adopt costly social solutions like stopping technological progress. (And actually I don’t expect the evidence to be highly persuasive either, so this amounts to just another small chance of high return.)
What I wrote in an earlier post still describes my overall position: