I agree that this is and should be a core goal of rationality. It’s a bit unclear to me how easy it would have been to predict the magnitude of the problem in advance. There’s a large number of things to get right when inventing a whole new worldview and culture from scratch. (Insofar as it was predictable in advance, I think it is good to do some kind of backprop where you try to figure out why you didn’t prioritize it, so that you don’t make that same mistake again. I’m not currently sure what I’d actually learn here)
Meanwhile, my impression is that once “actually a couple people have had psychotic breaks oh geez”, CFAR was reasonably quick to pivot towards prioritize avoiding that outcome (I don’t know exactly what went on there and it’s plausible that response time should have been faster, or in response to earlier warning signs).
But, part of the reason this is hard is that there isn’t actually a central authority here and there’s a huge inertial mass of people already excited about brain-tinkering that’s hard to pivot on a dime.
I agree that this is and should be a core goal of rationality. It’s a bit unclear to me how easy it would have been to predict the magnitude of the problem in advance. There’s a large number of things to get right when inventing a whole new worldview and culture from scratch. (Insofar as it was predictable in advance, I think it is good to do some kind of backprop where you try to figure out why you didn’t prioritize it, so that you don’t make that same mistake again. I’m not currently sure what I’d actually learn here)
Meanwhile, my impression is that once “actually a couple people have had psychotic breaks oh geez”, CFAR was reasonably quick to pivot towards prioritize avoiding that outcome (I don’t know exactly what went on there and it’s plausible that response time should have been faster, or in response to earlier warning signs).
But, part of the reason this is hard is that there isn’t actually a central authority here and there’s a huge inertial mass of people already excited about brain-tinkering that’s hard to pivot on a dime.