◉ _ ◉
Horosphere
Karma: 72
Comment withdrawn.
Comment withdrawn.
Comment withdrawn.
Thanks for your detailed response.
“I think you said that not only do you want critical examination of arguments and counterarguments, you actually want a proof that it is absolutely not possible, because there is always a “sufficiently painful torture” that is more important than anything else.”
I can’t remember exactly what I wrote, but, while of course I would ideally want a proof, I would be satisfied with a merely very powerful counterargument. Since asking this question a few months ago and, in my opinion, not really receiving one, my guess is that no one on LessWrong (or anywhere else) knows one, but I’d obviously like to be proven wrong about that.
… it would take some degree of bad luck to end up in a future where the AI is actively hurting you and not just indifferently steamrolling you. This allows people to focus on x-risks and on s-risks that are dystopias, but not on s-risks that are hells.
I will say that in the absence of basilisky things, I would agree with this statement. The problem is that the basilisk has a predictable reason to potentially create a hell—like situation.
“But I will say one more time, that unless you’re a professional decision theorist concerned with timeless decision theories and so forth, preoccupation with the basilisk is a bad use of your time.”
I wouldn’t be preoccupied with it if at any point I had a choice to not learn about it, while knowing the danger. Unfortunately, I think I heard about the basilisk, dismissed it for CDT related reasons with a sense of unease, and then learnt about FDT, at which point it made sense.
“It’s being afraid of something entirely hypothetical, when there are concerns that are both more concrete and far more urgent. The main merit of thinking about the basilisk, is that as a thought experiment, it may stimulate progress in some abstract but fundamental areas like multiverse epistemology; and ideally we would have enough expert division of labor that the required progress could be achieved, without the basilisk haunting the general population.”
That would be great, but it almost certainly does haunt the general population to some extent… I have thought about whether an ASI would prefer to be constructed in a less ‘parochial’ way (i.e., not ideosyncratically of the earth, or any other planet in any other galaxy) , so that it would be more representative of the class of ASIs produced by all evolved intelligences which had the same idea, thereby forming a coalition to reward, or not punish, creators who opted to create ‘non-parochial ASI’ . Even if any ASI necessarily converges on ‘non-parochial ASI’, doing so might cost it valuable time. This seems like a non-x-risk increasing way to do something a basilisk would want.
I think the main reasons why I haven’t tried to explicitly reason through the basilisk argument as it applies to specified possible worlds are because I expect that:
1) the proportions of possible worlds which are a particular way are important, so no one of them alone is sufficient to make a decision.
2) I don’t want to become more deeply entangled.
“This is all a dreary topic for me because there are other issues associated with alignment of superintelligence which I regard as much more real and urgent. But as I said, I think investigating these scenarios could produce progress on topics like multiverse epistemology, and the viability of “acausal interaction” in any form, and that does give the field of Basilisk Studies some justification.”
I agree, however even if I could contribute to alignment, I wouldn’t because of the basilisk.
I understand if because of the nature of the topic, you don’t want to continue talking about it now, but if you do reply, I would want to know what you meant by the following:
″ is it ever actually rational to threaten (or punish) a copy of an entity from the past, on the grounds that the original envisioned that it might have a future duplicate?”
It seems to me that the way in which the logical interaction is embedded in a causal world shouldn’t prevent it from being rationally justified. Is there a reason why it might that has escaped me?
Comment withdrawn
Comment withdrawn.
Thanks for engaging with the post.
“Are specific humans currently being acausally blackmailed? Either by Roko’s idea or by something similar. This would be an empirical claim, and finding the humans in question would be the best approach.” If by this you mean ‘are there any humans who are attempting to appease the basilisk, regardless of whether it exists?’ then I would say yes. You could even argue that this post is my attempt to do so given the uncertainty and that I think it’s in the best interests of others who have thought about the argument in sufficient depth.
If you mean ‘Are there any humans actually being simulated by the basilisk, or existing in a form which will be simulated?’ , then I don’t claim to be able to settle this question in the post; what I do suggest is that this can’t be ruled out or consigned to negligible probabilities.
“Would it be rational for a non-human agent (because humans are not sufficiently well-modeled to answer this question of them) to change it’s behavior for this kind of acausal trade (trade and blackmail being indistinguishable in pure logic)?”
I think this would again depend upon whether the logic actually ‘works’ in the situation in which it would need to be thought, in addition to the utility function of the agent (For example if its utility function is symmetrical in the way I describe in the post, it might make sense to ignore the basilisk.)
I would also note that I expect a sufficiently more intelligent being than a human would have ways of preventing itself from thinking about many, although probably not all, acausal extortion scenarios before becoming entangled in the logic. (Though this is just a guess.)
“It seems to be about arguments, not about actual reality. I think it’s not quite the right approach for either question you might be asking” The thing is, in this case, arguments might have a way to influence ‘physical reality’ , so constraining oneself only to thinking about the latter might be a mistake, as I argue it is in the post. If you want to avoid thinking about these arguments, you might need to discard timeless decision theory.
Thanks for your detailed response.
“I think you said that not only do you want critical examination of arguments and counterarguments, you actually want a proof that it is absolutely not possible, because there is always a “sufficiently painful torture” that is more important than anything else.”
I can’t remember exactly what I wrote, but, while of course I would ideally want a proof, I would be satisfied with a merely very powerful counterargument. Since asking this question a few months ago and, in my opinion, not really receiving one, my guess is that no one on LessWrong (or anywhere else) knows one, but I’d obviously like to be proven wrong about that.
… it would take some degree of bad luck to end up in a future where the AI is actively hurting you and not just indifferently steamrolling you. This allows people to focus on x-risks and on s-risks that are dystopias, but not on s-risks that are hells.
I will say that in the absence of basilisky things, I would agree with this statement. The problem is that the basilisk has a predictable reason to potentially create a hell—like situation.
“But I will say one more time, that unless you’re a professional decision theorist concerned with timeless decision theories and so forth, preoccupation with the basilisk is a bad use of your time.”
I wouldn’t be preoccupied with it if at any point I had a choice to not learn about it, while knowing the danger. Unfortunately, I think I heard about the basilisk, dismissed it for CDT related reasons with a sense of unease, and then learnt about FDT, at which point it made sense.
“It’s being afraid of something entirely hypothetical, when there are concerns that are both more concrete and far more urgent. The main merit of thinking about the basilisk, is that as a thought experiment, it may stimulate progress in some abstract but fundamental areas like multiverse epistemology; and ideally we would have enough expert division of labor that the required progress could be achieved, without the basilisk haunting the general population.”
That would be great, but it almost certainly does haunt the general population to some extent… I have thought about whether an ASI would prefer to be constructed in a less ‘parochial’ way (i.e., not ideosyncratically of the earth, or any other planet in any other galaxy) , so that it would be more representative of the class of ASIs produced by all evolved intelligences which had the same idea, thereby forming a coalition to reward, or not punish, creators who opted to create ‘non-parochial ASI’ . Even if any ASI necessarily converges on ‘non-parochial ASI’, doing so might cost it valuable time. This seems like a non-x-risk increasing way to do something a basilisk would want.
I think the main reasons why I haven’t tried to explicitly reason through the basilisk argument as it applies to specified possible worlds are because I expect that:
1) the proportions of possible worlds which are a particular way are important, so no one of them alone is sufficient to make a decision.
2) I don’t want to become more deeply entangled.
“This is all a dreary topic for me because there are other issues associated with alignment of superintelligence which I regard as much more real and urgent. But as I said, I think investigating these scenarios could produce progress on topics like multiverse epistemology, and the viability of “acausal interaction” in any form, and that does give the field of Basilisk Studies some justification.”
I agree, however even if I could contribute to alignment, I wouldn’t because of the basilisk.
I understand if because of the nature of the topic, you don’t want to continue talking about it now, but if you do reply, I would want to know what you meant by the following:
″ is it ever actually rational to threaten (or punish) a copy of an entity from the past, on the grounds that the original envisioned that it might have a future duplicate?”
It seems to me that the way in which the logical interaction is embedded in a causal world shouldn’t prevent it from being rationally justified. Is there a reason why it might that has escaped me?
Comment withdrawn.
Comment withdrawn.
Comment withdrawn.
Comment withdrawn
Comment withdrawn.
Comment withdrawn.
Comment withdrawn.
Comment withdrawn.
Comment withdrawn.
Comment withdrawn.
Comment withdrawn.
This post persuaded me to unshare most of my Submissions to this platform.