You can believe a falsity for sake of utility while alieving a truth for sake of sanity. Deep down you know you’re not the best golfer, but there’s no reason to critically analyze your delusions if believing so’s been shown time and time again to make you a better golfer. The problems occur when your occupation is ‘FAI programmer’ or ‘neurosurgeon’ instead of ‘golfer’. But most of us aren’t FAI programmers or neurosurgeons, we just want to actually turn in our research papers on time.
It’s not even really that dangerous, as rationalists can reasonably expect their future selves to update on evidence that their past-inherited beliefs aren’t getting them utility (aren’t true): by this theory, passive avoidance of rationality is epistemically safer than active doublethink (which might not even be possible, as Eliezer points out). If something forces you to really pay attention to your false belief then the active process of introspection will lead to it being destroyed by the truth.
Added: You know, now that I think about it more, the real distinction in question isn’t aliefs and beliefs but instead beliefs and beliefs in beliefs; at least that’s how it works when I introspect. I’m not sure if studies show that performance is increased by belief in belief or if the effect is limited to ‘real’ belief. Therefore my whole first paragraph above might be off-base; anyone know the literature? I just have the secondhand CliffsNote pop-psy version. At any rate the second paragraph still seems reasonably clever… which is a bad sign.
Double added: Mike Blume’s post indicates my first paragraph may not have been off the mark. Belief in belief seems sufficient for performance enhancement. Actually, as far as I can tell, Blume’s post really just kinda wins the debate. Also see JamesAndrix’s comment.
You can believe a falsity for sake of utility while alieving a truth for sake of sanity. Deep down you know you’re not the best golfer, but there’s no reason to critically analyze your delusions if believing so’s been shown time and time again to make you a better golfer. The problems occur when your occupation is ‘FAI programmer’ or ‘neurosurgeon’ instead of ‘golfer’. But most of us aren’t FAI programmers or neurosurgeons, we just want to actually turn in our research papers on time.
It’s not even really that dangerous, as rationalists can reasonably expect their future selves to update on evidence that their past-inherited beliefs aren’t getting them utility (aren’t true): by this theory, passive avoidance of rationality is epistemically safer than active doublethink (which might not even be possible, as Eliezer points out). If something forces you to really pay attention to your false belief then the active process of introspection will lead to it being destroyed by the truth.
Added: You know, now that I think about it more, the real distinction in question isn’t aliefs and beliefs but instead beliefs and beliefs in beliefs; at least that’s how it works when I introspect. I’m not sure if studies show that performance is increased by belief in belief or if the effect is limited to ‘real’ belief. Therefore my whole first paragraph above might be off-base; anyone know the literature? I just have the secondhand CliffsNote pop-psy version. At any rate the second paragraph still seems reasonably clever… which is a bad sign.
Double added: Mike Blume’s post indicates my first paragraph may not have been off the mark. Belief in belief seems sufficient for performance enhancement. Actually, as far as I can tell, Blume’s post really just kinda wins the debate. Also see JamesAndrix’s comment.