Thank you for the feedback. I’ll try to address the key points. 1) I actually have looked into EY’s metaphysical beliefs and my conclusion is they are inconsistent and opaque at best, and have been criticized for that here. In any case, when I say someone operates from a single metaphysical viewpoint like physicalism, this is not any kind of criticism of their inability to consider something properly or whatnot. It just tries to put things into wider context by explaining that changing the metaphysical assumptions might change their conclusions or predictions.
2) The post in no way says that there is something that would “prevent” the existential risk. It clearly states such risk would not be mitigated. I could have made this more explicit. What the post says is that by introducting a “possibility,” no matter how remote, of certain higher coordination or power that would attempt to prevent X-risk because it is not in its interest, then in such a universe the expected p(doom) would be lower. Does that make sense?
3) You say that
If you want to argue against physicalism, there’s a very simple, inarguable method that would prove it. All you need to do is find one single reproducible example, anywhere, ever, of any part of the universe behaving differently than the laws of physics say they should
My reaction to that is that here your are exactly conflating physicalism with the “descriptive scope of science” which is exactly the category mistake I’m trying to point to! There will always be something unexplainable beyond the descriptive scope of science, and physicalism is filling that with “nothing but more physical-like clockwork things.” But that is a belief. It might be the “most logical believe with fewest extra assumptions.” But that should not grant it any special treatment among all other metaphysical interpretations.
4) Yes, I used the word “share/transmit information across distance” while describing non-locality. And while you cannot “use entanglement to transmit information,” I think it’s correct to say that the internal information of an entangled particle transmits information of its internal state to its entangled partner?
5) Please, don’t treat this as an “attack on AI safety”—all I’m trying is to open it to a wider metaphysical consideration.
(1) Ok, fair enough, that wasn’t clear to me on first read. I do think it’s worth noting that he does, in fact, consider many other viewpoints before rejecting them, and gives clear explanations of his reasons for doing so, whether you agree or not. He also in many places discusses why he thinks introducing those other viewpoints does not actually help. Others in the community have since engaged with similar ideas from many other viewpoints.
(2) That conclusion does not follow from the premises. In particular, you have not considered the set of possible worlds where a higher coordination or power is attempting to increase x-risk or cause human extinction, something which is not exactly rare among human belief systems. As such, it is not clear to me in which direction this pushes the probability of human extinction. For example, human extinction or near-extinction happens twice in the Bible, multiple times in Norse and Greek mythology, cyclically in Buddhism and Hinduism, and some hard-to-determine fraction of the time in simulation hypotheses.
(3) This is not about science. It is about basic logic. There are two mutually exclusive and collectively exhaustive possibilities. One is that everything, everywhere, without exception, throughout the universe, behaves in accordance with some set of physical laws. This is physicalism. The other is the very expansive category of “everything else.” If anything within all of “everything else” is true, then there is somewhere in the universe that physicalism’s central claim does not hold. If you find even one such place, then scientists will investigate and ultimately concede the point. Otherwise, what exactly is it that you (or whoever else) are claiming to believe, and how did you select that belief out of the expansive set of all possibilities?
(4) This is also not correct. It sounds like what you are describing, essentially, are local hidden variables, which are mostly ruled out by Bell’s Theorem. There are known theories that get around this restriction, but as far as I know all of them look like strengthening the deterministic constraints on the behavior of all systems everywhere. They do not have anything to do with non-physicalist metaphysics. If you want to claim that the predictions of QM are coming from some non-physicalist metaphysical source, then technically I cannot rule that out, but you should realize this means that that source must be of such nature that it causes the universe to behave in extremely precise and mathematically consistent ways, which does not sound at all like what you want from this discussion. If anything, it sounds like a new law of physics.
(5) Sorry, I didn’t mean to treat it as such. It’s clear you are aware that x-risk is a thing worth caring about. I just thought it was worth highlighting that someone you cited, and a world leader in a category you cited, seem as though they are not presently on board with thinking that their philosophies or metaphysics reduce the probability of x-risk, or at least not enough to worry less about it.
2) Yes, that is true. I did leave out a sentence saying that “this assumes that there are no higher P(doom) realities in our list of plausible realities.” I left it our for readibility for the audience of the original publication (Phi/AI). I concede that for LW I should have done a more rigorous version.
But still I think the logic to lower our P(doom) holds in that specific analysis (all 3 alternatives might have some failsafes). And in my eyes it would hold also if we look at the current landscape of the top most plausible metaphysics, where there really is not much more “unsafe” than physicalism in terms of human survival.
3) I think you are not correct in your conclusions about physicalism. Physicalism is, by its proper definition, a philosophical belief: “Physicalism is the philosophical view that everything in existence is fundamentally physical, or at least ultimately depends on the physical, meaning there is “nothing over and above” the physical world.” This means that physicalism goes beyond the “simple logic” you described. The simple logic you described can only ever explain the parts of our reality that can be subjected to experimental observation—i.e. it’s limited by the descriptive scope of science. But physicalism goes beyond that by believing that there is nothing “extra” added beyond that.
For example, if our world would be a simulation with fixed rules (physical laws) run by an alien, your simple logic could not distinguish that from a scenario where our world just “popped up from nothing.” So the only “special place” physicalism holds among philosophical views is that it introduces the least amount of “extra assumptions.” But that says nothing about its ultimate plausibility.
Another way to picture this is that everytime we want to build a complete model of reality, there will be two parts: one verifiable by experiment (science) and the other inherently unverifiable (philosophy). The fact that physicalism is picking the “simplest, least complicated philosophical framework” should in no way lead us to ignore all the other, equally unverifiable, alternatives.
4) I am not the one originally making the claim that the experiments that proved non-locality of QM had profound implications on metaphysical and philosophical discourse. In the post, I link to the article “Enter experimental metaphysics” by Hans Busstra, which might help you understand the context.
5) This article is only the very introduction to my ultimate goal of exploring alternative metaphysical frameworks to find novel approaches to AI safety. I’m sure the rationale will be clearer as I release further articles and I warmly invite you to read the full series.
(2) I hope you’ll dig into this more in those future posts, because I think it is extremely non-obvious.
(3) Yes, I will concede that example, you’re right. For any observer in any possible world, there are an arbitrarily large number of larger universes within which it could be a perfect simulation, and these would be indistinguishable from the inside. This is a thing we cannot know, and the choice to then act as if those unknowable things don’t exist is an additional choice. I definitely did not think this was the kind of metaphysical context you were pointing towards, considering that your post is ultimately about how non-physicalist metaphysical assumptions should alter our expectations regarding possible future events in our world. I hope in future posts you’ll explain (a) why you think this kind of class of indistinguishable-in-principle worlds is interesting and useful to think about, and (b) how it relates to the topic of this post.
(4) Yes, I did read it, and didn’t find anything novel in it, though it was a good summary of a common viewpoint. I am aware you are not the originator of this viewpoint. I am also aware that QM (or more specifically, a set of beliefs about QM) has, indeed, had profound implications on metaphysical and philosophical discourse. This is because the pioneers of QM were confused about what they were learning; how could they not be? That’s what being a pioneer is. And they taught generations of physicists in ways that perpetuated that confusion without holding back technical progress in experimental physics. But the theorists have continued to make progress, and physics does know that e.g. none of this requires any kind of conscious observers or retrocausality or superluminal signal transmission. That’s a confusion that has persisted, even among many physicists (or at least, among their readily available analogies for describing things colloquially), not a part of the physics itself.
by introducting a “possibility,” no matter how remote, of certain higher coordination or power that would attempt to prevent X-risk because it is not in its interest, then in such a universe the expected p(doom) would be lower
This is :not-exhaustive:
We may as well introduce a possibility of some power (of any origin, that we had not seen, but not ruling out that we might see it sometime) that would attempt to bring about X-risk for humanity because it is in its interest.
Thank you for the feedback. I’ll try to address the key points.
1) I actually have looked into EY’s metaphysical beliefs and my conclusion is they are inconsistent and opaque at best, and have been criticized for that here. In any case, when I say someone operates from a single metaphysical viewpoint like physicalism, this is not any kind of criticism of their inability to consider something properly or whatnot. It just tries to put things into wider context by explaining that changing the metaphysical assumptions might change their conclusions or predictions.
2) The post in no way says that there is something that would “prevent” the existential risk. It clearly states such risk would not be mitigated. I could have made this more explicit. What the post says is that by introducting a “possibility,” no matter how remote, of certain higher coordination or power that would attempt to prevent X-risk because it is not in its interest, then in such a universe the expected p(doom) would be lower. Does that make sense?
3) You say that
My reaction to that is that here your are exactly conflating physicalism with the “descriptive scope of science” which is exactly the category mistake I’m trying to point to! There will always be something unexplainable beyond the descriptive scope of science, and physicalism is filling that with “nothing but more physical-like clockwork things.” But that is a belief. It might be the “most logical believe with fewest extra assumptions.” But that should not grant it any special treatment among all other metaphysical interpretations.
4) Yes, I used the word “share/transmit information across distance” while describing non-locality. And while you cannot “use entanglement to transmit information,” I think it’s correct to say that the internal information of an entangled particle transmits information of its internal state to its entangled partner?
5) Please, don’t treat this as an “attack on AI safety”—all I’m trying is to open it to a wider metaphysical consideration.
(1) Ok, fair enough, that wasn’t clear to me on first read. I do think it’s worth noting that he does, in fact, consider many other viewpoints before rejecting them, and gives clear explanations of his reasons for doing so, whether you agree or not. He also in many places discusses why he thinks introducing those other viewpoints does not actually help. Others in the community have since engaged with similar ideas from many other viewpoints.
(2) That conclusion does not follow from the premises. In particular, you have not considered the set of possible worlds where a higher coordination or power is attempting to increase x-risk or cause human extinction, something which is not exactly rare among human belief systems. As such, it is not clear to me in which direction this pushes the probability of human extinction. For example, human extinction or near-extinction happens twice in the Bible, multiple times in Norse and Greek mythology, cyclically in Buddhism and Hinduism, and some hard-to-determine fraction of the time in simulation hypotheses.
(3) This is not about science. It is about basic logic. There are two mutually exclusive and collectively exhaustive possibilities. One is that everything, everywhere, without exception, throughout the universe, behaves in accordance with some set of physical laws. This is physicalism. The other is the very expansive category of “everything else.” If anything within all of “everything else” is true, then there is somewhere in the universe that physicalism’s central claim does not hold. If you find even one such place, then scientists will investigate and ultimately concede the point. Otherwise, what exactly is it that you (or whoever else) are claiming to believe, and how did you select that belief out of the expansive set of all possibilities?
(4) This is also not correct. It sounds like what you are describing, essentially, are local hidden variables, which are mostly ruled out by Bell’s Theorem. There are known theories that get around this restriction, but as far as I know all of them look like strengthening the deterministic constraints on the behavior of all systems everywhere. They do not have anything to do with non-physicalist metaphysics. If you want to claim that the predictions of QM are coming from some non-physicalist metaphysical source, then technically I cannot rule that out, but you should realize this means that that source must be of such nature that it causes the universe to behave in extremely precise and mathematically consistent ways, which does not sound at all like what you want from this discussion. If anything, it sounds like a new law of physics.
(5) Sorry, I didn’t mean to treat it as such. It’s clear you are aware that x-risk is a thing worth caring about. I just thought it was worth highlighting that someone you cited, and a world leader in a category you cited, seem as though they are not presently on board with thinking that their philosophies or metaphysics reduce the probability of x-risk, or at least not enough to worry less about it.
2) Yes, that is true. I did leave out a sentence saying that “this assumes that there are no higher P(doom) realities in our list of plausible realities.” I left it our for readibility for the audience of the original publication (Phi/AI). I concede that for LW I should have done a more rigorous version.
But still I think the logic to lower our P(doom) holds in that specific analysis (all 3 alternatives might have some failsafes). And in my eyes it would hold also if we look at the current landscape of the top most plausible metaphysics, where there really is not much more “unsafe” than physicalism in terms of human survival.
3) I think you are not correct in your conclusions about physicalism. Physicalism is, by its proper definition, a philosophical belief: “Physicalism is the philosophical view that everything in existence is fundamentally physical, or at least ultimately depends on the physical, meaning there is “nothing over and above” the physical world.” This means that physicalism goes beyond the “simple logic” you described. The simple logic you described can only ever explain the parts of our reality that can be subjected to experimental observation—i.e. it’s limited by the descriptive scope of science. But physicalism goes beyond that by believing that there is nothing “extra” added beyond that.
For example, if our world would be a simulation with fixed rules (physical laws) run by an alien, your simple logic could not distinguish that from a scenario where our world just “popped up from nothing.” So the only “special place” physicalism holds among philosophical views is that it introduces the least amount of “extra assumptions.” But that says nothing about its ultimate plausibility.
Another way to picture this is that everytime we want to build a complete model of reality, there will be two parts: one verifiable by experiment (science) and the other inherently unverifiable (philosophy). The fact that physicalism is picking the “simplest, least complicated philosophical framework” should in no way lead us to ignore all the other, equally unverifiable, alternatives.
4) I am not the one originally making the claim that the experiments that proved non-locality of QM had profound implications on metaphysical and philosophical discourse. In the post, I link to the article “Enter experimental metaphysics” by Hans Busstra, which might help you understand the context.
5) This article is only the very introduction to my ultimate goal of exploring alternative metaphysical frameworks to find novel approaches to AI safety. I’m sure the rationale will be clearer as I release further articles and I warmly invite you to read the full series.
(5) I look forward to it.
(2) I hope you’ll dig into this more in those future posts, because I think it is extremely non-obvious.
(3) Yes, I will concede that example, you’re right. For any observer in any possible world, there are an arbitrarily large number of larger universes within which it could be a perfect simulation, and these would be indistinguishable from the inside. This is a thing we cannot know, and the choice to then act as if those unknowable things don’t exist is an additional choice. I definitely did not think this was the kind of metaphysical context you were pointing towards, considering that your post is ultimately about how non-physicalist metaphysical assumptions should alter our expectations regarding possible future events in our world. I hope in future posts you’ll explain (a) why you think this kind of class of indistinguishable-in-principle worlds is interesting and useful to think about, and (b) how it relates to the topic of this post.
(4) Yes, I did read it, and didn’t find anything novel in it, though it was a good summary of a common viewpoint. I am aware you are not the originator of this viewpoint. I am also aware that QM (or more specifically, a set of beliefs about QM) has, indeed, had profound implications on metaphysical and philosophical discourse. This is because the pioneers of QM were confused about what they were learning; how could they not be? That’s what being a pioneer is. And they taught generations of physicists in ways that perpetuated that confusion without holding back technical progress in experimental physics. But the theorists have continued to make progress, and physics does know that e.g. none of this requires any kind of conscious observers or retrocausality or superluminal signal transmission. That’s a confusion that has persisted, even among many physicists (or at least, among their readily available analogies for describing things colloquially), not a part of the physics itself.
This is :not-exhaustive:
We may as well introduce a possibility of some power (of any origin, that we had not seen, but not ruling out that we might see it sometime) that would attempt to bring about X-risk for humanity because it is in its interest.