Sorry, I was trying to be really careful as I was writing of not accusing you specifically of bad intentions, but obviously it’s hard in a conversation like this where you’re jumping between the meta and the object-level.
It’s important to distinguish a couple things.
1. Jessica and I were talking about people with negative intentions in the last two posts. I’m not claiming that you’re one of those people that is deliberately using this type of argument to cause harm.
2. I’m not claiming that it was the writing of those two posts that were harmful in the way we were talking about. I was claiming that the long post you wrote at the top of the thread where you made several analogies about your response, were exactly the sort of gray area situations where, depending on context, the community might decide to sacrifice it’s sacred value. At the same time, you were banking on the fact that it was a sacred value to say “even in this case, we would uphold the sacred value.” This has the same structure as the werewolf move mentioned above, and it was important for me to speak up, even if you’re not a werewolf.
people with negative intentions [...] deliberately
So, it’s actually not clear to me that deliberate negative intentions are particularly important, here or elsewhere? Almost no one thinks of themselves as deliberately causing avoidable harm, and yet avoidable harm gets done, probably by people following incentive gradients that predictably lead towards harm, against truth, &c. all while maintaining a perfectly sincere subjective conscious narrative about how they’re doing God’s work, on the right side of history, toiling for the greater good, doing what needs to be done, maximizing global utility, acting in accordance with the moral law, practicing a virtue which is nameless, &c.
it was important for me to speak up, even if you’re not a werewolf.
Agreed. If I’m causing harm, and you acquire evidence that I’m causing harm, then you should present that evidence in an appropriate venue in order to either persuade me to stop causing harm, or persuade other people to coördinate to stop me from causing harm.
I was claiming that the long post you wrote at the top of the thread where you made several analogies about your response, were exactly the sort of gray area situations where, depending on context, the community might decide to sacrifice it’s sacred value.
So, my current guess (which is only a guess and which I would have strongly disagreed with ten years ago) is that this is a suicidally terrible idea that will literally destroy the world. Sound like an unreflective appeal to sacred values? Well, maybe!—you shouldn’t take my word for this (or anything else) except to the exact extent that you think my word is Bayesian evidence. Unfortunately I’m going to need to defer supporting argumentation to future Less Wrong posts, because mental and financial health requirements force me to focus on my dayjob for at least the next few weeks. (Oh, and group theory.)
So, it’s actually not clear to me that deliberate negative intentions are particularly important, here or elsewhere?
(responding, and don’t expect another response back because you’re busy).
I used to think this, but I’ve since realized that intentions STRONGLY matter. It seems like a system is fractal, the goals of the subparts/subagents get reflected in the goal of the broader system. People with aligned intentions will tend to shift the incentive gradients, as well people with unaligned intentions (of course, this isn’t a one way relationship, the incentive gradients will also shift the intentions).
Sorry, I was trying to be really careful as I was writing of not accusing you specifically of bad intentions, but obviously it’s hard in a conversation like this where you’re jumping between the meta and the object-level.
It’s important to distinguish a couple things.
1. Jessica and I were talking about people with negative intentions in the last two posts. I’m not claiming that you’re one of those people that is deliberately using this type of argument to cause harm.
2. I’m not claiming that it was the writing of those two posts that were harmful in the way we were talking about. I was claiming that the long post you wrote at the top of the thread where you made several analogies about your response, were exactly the sort of gray area situations where, depending on context, the community might decide to sacrifice it’s sacred value. At the same time, you were banking on the fact that it was a sacred value to say “even in this case, we would uphold the sacred value.” This has the same structure as the werewolf move mentioned above, and it was important for me to speak up, even if you’re not a werewolf.
Thanks for clarifying!
So, it’s actually not clear to me that deliberate negative intentions are particularly important, here or elsewhere? Almost no one thinks of themselves as deliberately causing avoidable harm, and yet avoidable harm gets done, probably by people following incentive gradients that predictably lead towards harm, against truth, &c. all while maintaining a perfectly sincere subjective conscious narrative about how they’re doing God’s work, on the right side of history, toiling for the greater good, doing what needs to be done, maximizing global utility, acting in accordance with the moral law, practicing a virtue which is nameless, &c.
Agreed. If I’m causing harm, and you acquire evidence that I’m causing harm, then you should present that evidence in an appropriate venue in order to either persuade me to stop causing harm, or persuade other people to coördinate to stop me from causing harm.
So, my current guess (which is only a guess and which I would have strongly disagreed with ten years ago) is that this is a suicidally terrible idea that will literally destroy the world. Sound like an unreflective appeal to sacred values? Well, maybe!—you shouldn’t take my word for this (or anything else) except to the exact extent that you think my word is Bayesian evidence. Unfortunately I’m going to need to defer supporting argumentation to future Less Wrong posts, because mental and financial health requirements force me to focus on my dayjob for at least the next few weeks. (Oh, and group theory.)
(End of thread for me.)
(responding, and don’t expect another response back because you’re busy).
I used to think this, but I’ve since realized that intentions STRONGLY matter. It seems like a system is fractal, the goals of the subparts/subagents get reflected in the goal of the broader system. People with aligned intentions will tend to shift the incentive gradients, as well people with unaligned intentions (of course, this isn’t a one way relationship, the incentive gradients will also shift the intentions).