People only receive feedback from people that are engaged enough to give it.
On The Internet, that’s generally true. But that’s not so true IRL, face-to-face. And the point of the post is that we could engineer feedback-by-default like the reactions people mostly can’t help having when they’re visible (or audible) in small groups.
I agree that there might not be anything wrong with supporting a specific X without also supporting (or with opposing) all X in general. But that all depends on the reasons why you support the specific X but don’t support (or oppose) the general X. Why did you sign the petition but the general policy? (Also, what do you think the general policy is exactly?)
I don’t personally have strong feelings or convictions pertaining to all of this. I don’t want the NYT to publish Scott’s full legal name, but I don’t have any particular strong objections about them or anyone else doing that in general. I do oppose the specific politics that I think is motivating them publishing his name. I also don’t think there are any good reasons to publish his name that aren’t motivated to hurt or harm him.
You signed the position purely out of instrumental concerns and any principles about petitions and how news organizations should or should not respond to them is entirely independent? Admitting that – even judged just instrumentally – seems counter-productive.
The relevant principle seems pretty clear (to me): of course people should be generally open to being swayed by (reasoned) argumentation, e.g. via petition – unless there’s some concern(s) that override it, like a principled pre-commitment to ignore some types of influence (for very good reasons).
Thanks again!
Your point about “Bayesianism at a distance” makes a lot of sense.
Thanks! That answers a lot of my questions even without a concrete example.
I found this part of your reply particularly interesting:
if you don’t have (2), updates are not very constrained by Dutch-book type rationality. So in general, Jeffrey argued that there are many valid updates beyond Bayes and Jeffrey updates.
The abstract example I came up with after reading that was something like ‘I think A at 60%. If I observe X, then I’d update to A at 70%. If I observe Y, then I’d update to A at 40%. If I observe Z, I don’t know what I’d think.’.
I think what’s a little confusing is that I imagined these kinds of adjustments were already incorporated into ‘Bayesian reasoning’. Like, for the canonical ‘cancer test result’ example, we could easily adjust our understanding of ‘receives a positive test result’ to include uncertainty about the evidence itself, e.g. maybe the test was performed incorrectly or the result was misreported by the lab.
Do the ‘same’ priors cover our ‘base’ credence of different types of evidence? How are probabilities reasonably, or practically, assigned or calculated for different types of evidence? (Do we need to further adjust our confidence of those assignment or calculations?)
Maybe I do still need a concrete example to reach a decent understanding.
Are there any other detailed descriptions of what a “Jeffrey update” might look like or how one would perform one?
I think I get the point of there being “rationality constraints” that don’t, by implication, strictly require Bayesian updates. But are Jeffrey updates the entire set of possible updates that are required?
Can anyone describe a concrete example contrasting a Bayesian update and a Jeffrey update for the same circumstances, e.g. prior beliefs and new information learned?
It kinda seems like Jeffrey updates are ‘possibly rational updates’ but they’re only justified if one can perform them for no possible (or knowable) reason. That doesn’t seem practical – how could that work?
I can’t think of a way that could work that couldn’t be automated away, e.g. to a barrier consisting solely of ‘install this browser extension’. (Or not at least without being a relatively non-trivial annoyance to the ‘trusted’ users too.)
I think there’s a way to submit feature requests to the LW dev(s). An RSS feed for ‘Recent Discussion’ should be possible.
And, as I wrote before, I sometimes look at posts on their original sites – LW is one for which I do this a lot of the time (e.g. so I can vote or comment).
Users on this “network” are capable of being pseudonymous. Anonymity is probably also possible, tho (much?) harder. We don’t seem to have attracted too many people “spewing nonsense”, or that many at all.
Requiring a personal connection to existing users will shut out a lot of potential users. And it’s probably better for plausible deniability that we continue to allow anyone to signup.
I – and I’d guess most other users – are not doing enough to reliably avoid de-anonymization. It requires very strict opsec in general.
And I don’t know how you could possibly calculate the probability of being de-anonymized, even with perfect information about everything you’ve leaked. Relying on your feelings is probably the only practical option, besides not sharing any info.
Is a commitment to entertain controversial or unpopular or odious ideas (or to advocate for them) separate from or integral to rationalism?
Integral – for epistemological rationality anyways, and arguably too for instrumental rationality as well.
Is a mental health professional’s preference to maintain enough anonymity so that their blog does not interfere with their practice or their safety separate from or integral to rationality?
I don’t think it’s “separate from” as much as ‘mostly orthogonal’. Scott is largely to blame for his relative lack of pseudonymity – he’s published, publicly, a lot of evidence of his identity. What’s he trying to avoid is losing enough of what remains so that his (full) legal name is directly linked to Scott Alexander – in the top results of, e.g. a Google search result for his legal name.
When it comes to the general idea that anonymity is needed to discuss certain or any topics, I’m more skeptical.
You’re right, it’s not needed to discuss anything – at least not once. The entire issue is whether one can do so indefinitely. And, in that case, it sure seems like anonymity/pseudonymity is needed, in general.
I don’t think there’s a lot of anonymity here on LessWrong but it’s certainly possible to be pseudonymous. I don’t think most people bother to really try to maintain it particularly strictly. But I find the comments here to be much better than in anonymous/pseudonymous comments in other places, or even – as you seem to agree – on or via FaceBook or Twitter (or whatever). This place is special. And I think this place really is vulnerable to censorship, i.e. pressure NOT to discuss what’s discussed here now. The people here – some of them anyways – really would refrain from discussing some things were they to suffer for it like they fear.
There’s no purely technological solution to censorship, especially indirect forms like what’s arguably happening to Scott.
Bruce Schneier, a famous cryptography expert and advocate, eventually realized that cryptography, while good, was often besides the point – it almost always made sense to attack the other parts of systems instead of try to break encryption algorithms. Here’s a good essay by him about this and a matching XKCD:
To the degree that we want to avoid censorship, we need to either adopt sufficient opsec – much better than Scott and even better than Gwern, who wrote to someone blackmailing them (by threatening to dox them):
It would be annoying to have my name splashed all over, but I resigned myself to that back in ~2010 when I decided to set up my website; I see Gwern as a pen name now, and not a real pseudonym. I’m glad it managed to last to 2015.
– or we need to prevent censorship IRL. Obviously, we can (and should) do both to some extent.
But really, to avoid the kind of censorship that inspired this, one would need to remain strictly anonymous/pseudonymous, which is hard (and lonely – IRL meetups are a threat vector!).
I don’t think that would work because a definite identity is needed for people to follow Scott. I don’t think I could possibly track ‘Scott’, or anyone, and notice that there was a specific identity, if I couldn’t track a named identity, i.e. a specific account.
Part of who Scott is to a lot of us is someone, i.e. a specific identity, that’s worth following, tracking, or paying attention to over long periods. Using throwaway accounts makes that harder, even ignoring the kind of fuckery that someone could pull with something like GPT-3 at hand – maybe it’s even (effectively) impossible now.
And to the degree that we all could continue to track Scott, even with him posting via throwaway accounts – and even assuming otherwise perfect opsec from Scott too – so could people who feel malice towards him track him similarly. We’d see ‘likely Scott posts’ lists. We wouldn’t be able to prevent people from speculating as to which posts were his. Scott would only be protected to the degree that we couldn’t tell it was him.
There’s probably a math/compsci/physics theory that covers this, but there’s no way for Scott to be able to maintain both his pseudonymity and his identity – to the degree he wants (or needs) – if his (full) legal name is linked to his pseudonym on the first page of web search results for the legal name.
The safer option would be for Scott to create a new pseudonym and try to maintain (very) plausible deniability to any connection to Scott Alexander. But that would mean starting over and would probably be very hard – and maybe impossible anyways. It might be impossible for anyone to write in a different enough style, consistently, and avoid detection, especially at the lengths he typically posts. It’s probably even that much harder (or impossible) for people that have written as much as he has publicly.
Are you sure RSS won’t work for you? I follow LW and a bunch of other sites/feeds in a feed reader (Feedly). I scan all of my feeds in a big ‘oldest first’ list, read most in the reader, but sometimes look at posts on their original sites (via a keyboard shortcut consisting of a single key) – the latter seems like a perfectly fine way to read comments (when I want to).
Or do you need or want to follow comments too, i.e. comments made after you read the original post? I think a lot of sites have comments feeds, so that should be possible via RSS too. I don’t do this tho. I do selectively subscribe to comments for specific posts, but typically via email. On some sites, including LW, I only get new comments via on-site or in-app notifications, which mostly seems fine.
Anonymity is hard – the posts themselves, e.g. word choice, phrasing, etc., are all extremely vulnerable ‘side channels’ for breaking anonymity. Defending against those kinds of attacks are probably both impractical and extremely difficult, if not effectively impossible.
The most important step in defending against attacks is (at least roughly) determining a threat model. What kind of attacks do you expect? What attacks are worth defending against – and which aren’t?
My point about ‘capping’ the (dis)utility of pain was that one – a person or mind that isn’t a malevolent (super-)intelligence – wouldn’t want to be able to be ‘held hostage’ were something like a malevolent super-intelligence in control of some other mind that could experience ‘infinite pain’. You probably wouldn’t want to sacrifice everything for a tiny chance at preventing the torture of a single being, even if that being was capable of experiencing infinite pain.
I don’t think it’s possible, or even makes sense, for a mind to experience an infinite amount/level/degree of pain (or suffering). Infinite pain might be possible over an infinite amount of time, but that seems (at least somewhat) implausible, e.g. given that the universe doesn’t seem to be infinite, seems to contain a finite amount of matter and energy, and seems likely to die of an eventual heat death (and thus not able to support life or computation indefinitely).
Even assuming that a super-intelligence could rewire human minds to just increase the amount of pain they can experience, a reasonable generalization is to a super-intelligence creating (e.g. simulating) minds (human or otherwise). That seems to me to be the same (general) moral/ethical catastrophe as your hypothetical(s).
But I don’t think these hypotheticals really alter the moral/ethical calculus with respect to our decisions, i.e. the possibility of the torture of minds that can experience infinite pain doesn’t automatically imply that we should avoid developing AGI or super-intelligences entirely. (For one, if infinite pain is possible, so might infinite joy/happiness/satisfaction.)
(You need a space between the > and the text being quoted to format it as a quote in Markdown.)
Sure, we can assume a malevolent super-intelligence could prevent people from going into shock and thus cause much more pain than otherwise.
But it’s not clear how (or even whether) we can quantize pain (or suffering). From the perspective of information processing (or something similar), it seems like there would probably be a maximum amount of non-disabling pain, i.e. a ‘maximum priority override’ to focus all energy and other resources on escaping that pain as quickly as possible. It also seems unclear why evolution would result in creatures able to experience pain more intensely than such a maximum.
Let’s assume pain has no maximum – I’d still expect a reasonable utility function to cap the (dis)utility of pain. If it didn’t, the (possible) torture of just one creature capable of experiencing arbitrary amounts/degrees/levels of pain would effectively be ‘Pascal’s hostage’ (something like, under the control of a malevolent super-intelligence, a utility monster).
But yes, a malevolent super-intelligence, or even just one that’s not perfectly ‘friendly’, would be terrible and the possibility is incredibly scary to me too!
Presumably pain works in some specific way (or some relatively narrow distribution of ways), so there probably is a maximum amount of pain that could be experienced in any circumstance. Real-life animals can and do die of shock, which seems like it might be some maximum ‘pain’ threshold being exceeded.
But suffering seems much more general than pain. Creating (e.g. simulating) a consciousness or mind and torturing it increases global suffering. Creating multiple minds and torturing them would increase suffering further.
What seems to be different about suffering (to at least some degree – real-life beings also seem to suffer sympathetic pain) is that additional suffering can be, in effect, created simply by informing other minds of suffering of which they were not previously aware. Some suffering is created by knowledge or belief, i.e. spread by information. (This post has a good perspective one can adopt to avoid being ‘mugged’ by this kind of information.)
The creation or simulation of minds is presumably bounded by physical constraints, thus there probably is some maximum amount of suffering possible.
Are there possible minds that can experience an infinite amount of pain or suffering? I think not. At a ‘gears’ level, it doesn’t seem like pain or suffering could literally ever be infinite, even over an infinite span of time, tho I admit that seems true because it does seem true that, e.g. there’s a finite amount of matter in the universe, and that minds cannot exist for an infinite amount of time (e.g. because of the eventual heat death of the universe).
But even assuming minds can exist for an infinite amount of time or that minds could be arbitrarily ‘large’, I’d expect the amount of pain or suffering that any one mind could experience to be finite. But, under those same assumptions (or similar ones), the total amount of pain or suffering experienced could be infinite.
I don’t think I’ve seen this idea being used in different ways.
Wait a few years and Dwarf Fortress might have implemented those gears.
I don’t think the post was about LessWrong specifically (at all); think Twitter or Facebook or random blog comments.
Here on this site, yes both downvotes and the absence of upvotes are strong mostly-legible signals.