Curated. I think it’s long been a problem that LessWrong doesn’t have great models on how to handle adversarial situations. I’ve been wanting Habryka to write up their thoughts on this for awhile.
I think this post is more like re-establishing some existing concepts that have already been in the water supply (as opposed to adding something new), but it does a good job introducing them in a way that sets up the problem, with a kind of practical mindset. It does a good job motivating why you’d want a more fleshed out model for thinking about this, and, I think a decent job at conveying some default options people have in today’s world, and why those options aren’t sufficient.
I was glad to see some comments discussing “how would we build a formal epistemology, that explicitly incorporates adversarial action? What’s the current state of the art, and what are the obstacles to moving forward with that?”
I think the penultimate paragraph “Do not be the kind of actor that forces other people to be paranoid” is very important. I maybe wish it got a bit more signposting. I’m guessing/hoping there will be future posts digging more into it. I think for this essay, it’d be nice if that part had a section header that at least made that final takeaway stand out a bit more.
Curated. I think it’s long been a problem that LessWrong doesn’t have great models on how to handle adversarial situations. I’ve been wanting Habryka to write up their thoughts on this for awhile.
I think this post is more like re-establishing some existing concepts that have already been in the water supply (as opposed to adding something new), but it does a good job introducing them in a way that sets up the problem, with a kind of practical mindset. It does a good job motivating why you’d want a more fleshed out model for thinking about this, and, I think a decent job at conveying some default options people have in today’s world, and why those options aren’t sufficient.
I was glad to see some comments discussing “how would we build a formal epistemology, that explicitly incorporates adversarial action? What’s the current state of the art, and what are the obstacles to moving forward with that?”
I think the penultimate paragraph “Do not be the kind of actor that forces other people to be paranoid” is very important. I maybe wish it got a bit more signposting. I’m guessing/hoping there will be future posts digging more into it. I think for this essay, it’d be nice if that part had a section header that at least made that final takeaway stand out a bit more.