Moving the conversation here between @yams and me here because I want to stop cluttering up Wei Dai’s shortform.
My take on this commentis that I do agree that Forethought isn’t doing the original version of metaphilosophy, and it’s my bad that I’ve mentioned them. I’d say they’re doing perhaps the closest related work, which is still relevant, but yeah I don’t think that they are doing the original version of metaphilosophy, so I’ll strike out part of my comment on Forethought doing this.
On this part though:
Narrowing scope prematurely forces you back into the frame of those strategies you’re hoping to generate some alternative to.
If the initial scope of metaphilosophy wasn’t “solve the entire future”, I’d maybe agree with this, but in this case, it at least isn’t clear that the scope isn’t this large, and if it is in fact the case then I deny that this is what will happen by narrowing the scope.
The way I saw it, the scope was so large that narrowing it down was only going to cut down on strategies that would not work (or at least would not work without ASI), without disallowing too much creative freedom, because you can try to generate alternate strategies.
That said, I do think you have revealed a big crux in how I’m thinking about philosophy in terms of making the future go well, which might be useful to discuss.
[feels like a decent time to mention that I dislike this term quite a bit]
I don’t feel it’s that important to change terms for the sake of this conversation; I just think that ‘metaphilosophy’ is, miraculously, an even less useful term than its parent (‘philosophy’), and that I don’t necessarily want to talk about a relationship between the two concepts (esp. because the etymology is likely quite misleading).
There are many different levels of abstraction. Some of these are decently well-charted, and others aren’t. Sometimes to come to a new idea, you have to wade through the uncharted levels of abstraction and then, ultimately, take responsibility for making your idea defensibly touch reality once you’ve alighted upon something promising.
“Yes, this is just how generating novel ideas works,” I can hear someone saying.
Yes, but there’s a big difference in how far you let yourself go out before you reel it back in, and that project itself, of going further simply because no one else is going that far, because maybe you can see some new things out on that particular branch, is extremely worthwhile, and I will always defend a few brave souls taking this approach to any given problem (while the rest of us do something more tractable). I basically expect our understanding of reality has a few more Copernican revolutions left in it, and our local would-be stargazers ought to be encouraged, in particular when they have strong track records (as Wei Dai does).
This is what I mean by your call for tractability answering a completely different question than this inquiry tasks itself with. A bunch of people are doing a bunch of things that (to me) don’t seem to be getting us what we want (I’d include the Forethought work here); maybe someone should take an ambitious central question and push really hard on it, until something new comes about in them, that no one else has taken the risk to encounter before.
(e.g. thinking about AI a bunch precipitated innovations in, at least, decision theory, which would not necessarily be a completely obvious kind of innovation to expect from that inquiry)
Earlier I mentioned not loving Wei Dai’s approach; I think I want to say more about that, to help clarify the reference class I intend to defend (instead of just defending his position directly). I think that it’s a mistake to frame this inquiry as an attempt to ‘solve philosophy’; my best guess is that the existing questions are confused such that solutions don’t look anything like what we’d guess they look like from our current vantage point, that there’s some dissolution one should expect to occur, which will bring about new and different kinds of problems, while handling some of the old ones (and actually be better at relating to the world, by the way; I don’t mean to say that I think we just swap old problems for new equally-bad problems). I don’t see enough people picking up genuinely new tools or frames to think and talk about The Thing That May Be About to Happen, and that’s a big cause for my despair at the situation.
Mostly I think progress is made in a domain by throwing a toolkit at it at all, and not by endlessly elaborating on that same narrow toolkit. Instead, look for what the tools can’t handle, define a new tool kit around those exceptions, and see if that toolkit survives backtesting against the old problems, too. Then you’ve got a worthy successor paradigm, can onboard people into deploying it, and humanity learns a bunch of new stuff for a few decades. You get a new ‘way things are assumed to be’, which slowly reveals its own inadequacy over the special set of problems it engenders/fails against, and you iterate.
Some people say alignment is preparadigmatic; others say alignment has a paradigm, and it’s all this prosaic ML stuff; I think the real thing is ‘we have a paradigm, and it appears to be inadequate; let’s get a better paradigm.’
(I’m conflating between alignment and metaphilosophy-as-I-see-it, because I came to similar conclusions to Wei from reading a few hundred ML AI safety papers and a few tens of agent foundations writings; that is, I think the inquiry into alignment+++ (the strong MIRI-ish thing) itself is continuous with ‘metaphilosophy’, so framed.)
Moving the conversation here between @yams and me here because I want to stop cluttering up Wei Dai’s shortform.
My take on this comment is that I do agree that Forethought isn’t doing the original version of metaphilosophy, and it’s my bad that I’ve mentioned them. I’d say they’re doing perhaps the closest related work, which is still relevant, but yeah I don’t think that they are doing the original version of metaphilosophy, so I’ll strike out part of my comment on Forethought doing this.
On this part though:
If the initial scope of metaphilosophy wasn’t “solve the entire future”, I’d maybe agree with this, but in this case, it at least isn’t clear that the scope isn’t this large, and if it is in fact the case then I deny that this is what will happen by narrowing the scope.
The way I saw it, the scope was so large that narrowing it down was only going to cut down on strategies that would not work (or at least would not work without ASI), without disallowing too much creative freedom, because you can try to generate alternate strategies.
That said, I do think you have revealed a big crux in how I’m thinking about philosophy in terms of making the future go well, which might be useful to discuss.
Fair enough. What term would you like to be used?
I don’t feel it’s that important to change terms for the sake of this conversation; I just think that ‘metaphilosophy’ is, miraculously, an even less useful term than its parent (‘philosophy’), and that I don’t necessarily want to talk about a relationship between the two concepts (esp. because the etymology is likely quite misleading).
There are many different levels of abstraction. Some of these are decently well-charted, and others aren’t. Sometimes to come to a new idea, you have to wade through the uncharted levels of abstraction and then, ultimately, take responsibility for making your idea defensibly touch reality once you’ve alighted upon something promising.
“Yes, this is just how generating novel ideas works,” I can hear someone saying.
Yes, but there’s a big difference in how far you let yourself go out before you reel it back in, and that project itself, of going further simply because no one else is going that far, because maybe you can see some new things out on that particular branch, is extremely worthwhile, and I will always defend a few brave souls taking this approach to any given problem (while the rest of us do something more tractable). I basically expect our understanding of reality has a few more Copernican revolutions left in it, and our local would-be stargazers ought to be encouraged, in particular when they have strong track records (as Wei Dai does).
This is what I mean by your call for tractability answering a completely different question than this inquiry tasks itself with. A bunch of people are doing a bunch of things that (to me) don’t seem to be getting us what we want (I’d include the Forethought work here); maybe someone should take an ambitious central question and push really hard on it, until something new comes about in them, that no one else has taken the risk to encounter before.
(e.g. thinking about AI a bunch precipitated innovations in, at least, decision theory, which would not necessarily be a completely obvious kind of innovation to expect from that inquiry)
Earlier I mentioned not loving Wei Dai’s approach; I think I want to say more about that, to help clarify the reference class I intend to defend (instead of just defending his position directly). I think that it’s a mistake to frame this inquiry as an attempt to ‘solve philosophy’; my best guess is that the existing questions are confused such that solutions don’t look anything like what we’d guess they look like from our current vantage point, that there’s some dissolution one should expect to occur, which will bring about new and different kinds of problems, while handling some of the old ones (and actually be better at relating to the world, by the way; I don’t mean to say that I think we just swap old problems for new equally-bad problems). I don’t see enough people picking up genuinely new tools or frames to think and talk about The Thing That May Be About to Happen, and that’s a big cause for my despair at the situation.
Mostly I think progress is made in a domain by throwing a toolkit at it at all, and not by endlessly elaborating on that same narrow toolkit. Instead, look for what the tools can’t handle, define a new tool kit around those exceptions, and see if that toolkit survives backtesting against the old problems, too. Then you’ve got a worthy successor paradigm, can onboard people into deploying it, and humanity learns a bunch of new stuff for a few decades. You get a new ‘way things are assumed to be’, which slowly reveals its own inadequacy over the special set of problems it engenders/fails against, and you iterate.
Some people say alignment is preparadigmatic; others say alignment has a paradigm, and it’s all this prosaic ML stuff; I think the real thing is ‘we have a paradigm, and it appears to be inadequate; let’s get a better paradigm.’
(I’m conflating between alignment and metaphilosophy-as-I-see-it, because I came to similar conclusions to Wei from reading a few hundred ML AI safety papers and a few tens of agent foundations writings; that is, I think the inquiry into alignment+++ (the strong MIRI-ish thing) itself is continuous with ‘metaphilosophy’, so framed.)