You may be thinking of this exchange, which I found only because I remembered having been involved in it.
I continue to think that “tool” is a bad term to use here, because people’s understanding of what it refers to vary so relevantly.
As for what is valuable work… hm.
I think teaching people to reason in truth-preserving and value-preserving ways is worth doing. I think formalizing a decision theory that captures universal human intuitions about what the right thing to do in various situations is worth doing. I think formalizing a decision theory that captures non-universal but extant “right thing” intuitions is potentially worth doing, but requires a lot of auxiliary work to actually be worth doing. I think formalizing a decision theory that arrives at judgments about the right thing to do in various situations where those judgments are counterintuitive for most/all humans but reliably lead, if implemented, to results that those same humans reliably endorse more the results of their intuitive judgments is worth doing. I think building systems that can solve real-world problems efficiently is worth doing, all else being equal, though I agree that powerful tools frequently have unexpected consequences that create worse problems than they solve, in which case it’s not worth doing. I think designing frameworks within which problem-solving systems can be built, such that the chances of unexpected negative consequences are lower inside that framework than outside of it, is worth doing.
I don’t find it likely that SI is actually doing any of those things particularly more effectively than other organizations.
Thanks for the link—that was what I was thinking of.
Do you have other organizations which teach rationality in mind? Offhand, the only thing I can think of is cognitive behavioral therapy, and it’s not exactly an organization.
You may be thinking of this exchange, which I found only because I remembered having been involved in it.
I continue to think that “tool” is a bad term to use here, because people’s understanding of what it refers to vary so relevantly.
As for what is valuable work… hm.
I think teaching people to reason in truth-preserving and value-preserving ways is worth doing.
I think formalizing a decision theory that captures universal human intuitions about what the right thing to do in various situations is worth doing.
I think formalizing a decision theory that captures non-universal but extant “right thing” intuitions is potentially worth doing, but requires a lot of auxiliary work to actually be worth doing.
I think formalizing a decision theory that arrives at judgments about the right thing to do in various situations where those judgments are counterintuitive for most/all humans but reliably lead, if implemented, to results that those same humans reliably endorse more the results of their intuitive judgments is worth doing.
I think building systems that can solve real-world problems efficiently is worth doing, all else being equal, though I agree that powerful tools frequently have unexpected consequences that create worse problems than they solve, in which case it’s not worth doing.
I think designing frameworks within which problem-solving systems can be built, such that the chances of unexpected negative consequences are lower inside that framework than outside of it, is worth doing.
I don’t find it likely that SI is actually doing any of those things particularly more effectively than other organizations.
Thanks for the link—that was what I was thinking of.
Do you have other organizations which teach rationality in mind? Offhand, the only thing I can think of is cognitive behavioral therapy, and it’s not exactly an organization.
No, I don’t have anything specific in mind.