“Come back when your scientific journal is much later than 1880” is not particularly distinguishable, no.
Multiple things that I think are simultaneously true:
Something can satisfy all my pre-stated rejections and still not actually be good evidence for changing my mind. If that happens the right move is not to change my mind, because I don’t have good evidence.
There is a skill (or a cluster of related skills) which Is That Your True Rejection points at. Mastery of this skill is the ability to consider a belief of my own, concisely state the evidence which would change my mind on that belief, and be correct such that if that evidence is presented then my mind changes.
“I don’t like working out because I hate people watching me” is loosely isomorphic here to “I believe gravity on Earth accelerates objects down at about 9.8 m/s^2 because I dropped some objects off a high place and timed them.” The skill is pretty similar.
If someone points something out that satisfies my stated rejections, and I don’t change my mind, it’s pretty reasonable for them to be annoyed about that. I’m the one who failed a skill check.
If this keeps happening, where every time I try to pre-state my rejections and someone points out a way to satisfy them I find that I’m not convinced and add another reason, then I am not very good at the skill which Is That Your True Rejection points at.
There’s an unhelpful kind of munchkinry that looks for poor phrasing in pre-stated rejections like a devil in a fantasy novel or a highly motivated and perhaps unscrupulous lawyer, making suggestions that technically satisfy my rejections but obviously aren’t what I meant. There’s also a helpful kind of munchkinry that looks for out-of-the-box ways to work out without tripping anything we hate. The line is kind of blurry and may just outright be a gradient. I can’t write out rules that unhelpful munchkins couldn’t get around, but sometimes I know it when you see it and I don’t want to discourage helpful munchkinry.
Since the True Rejection Challenge exercise is in large part designed to practice the skill which Is That Your True Rejection points at, it’s going to involve a bunch of “oops, I didn’t think to add this rejection” in the beginning as well as the harder to spot failure of “oops, I added this as a rejection but it isn’t actually important to me.” When you start practicing juggling, you drop a lot of balls. That’s fine! Dropping balls in juggling practice and singing off key in choir practice aren’t mortal sins, they’re kind of supposed to happen! Obviously try not to make mistakes but go ahead and make lots of mistakes if it’s on the path to getting good at the skill.
In the case of prayer regrowing a leg, I think it’s possible someone failed a skill check by leaving the word “reputable” under-specified, and also possible their interlocutor is being an unhelpful munchkin. Both could be true at once! The helpful thing might be to say “hey, what do you mean by reputable?” Does that make sense?
There’s an unhelpful kind of munchkinry that looks for poor phrasing in pre-stated rejections like a devil in a fantasy novel or a highly motivated and perhaps unscrupulous lawyer, making suggestions that technically satisfy my rejections but obviously aren’t what I meant.
I think the original true rejection post is one of these. “Why should I believe him when he doesn’t have a PhD” probably means that the PhD is one of multiple conditions, not that it’s the sole reason for rejection. But Eliezer’s original post treats it as if it’s a gotcha.
Also, remember the Hidden Complexity of Wishes. Phrasing your rejection in such a way that nobody can abuse a misphrasing is really hard, even with feedback, and a lot like trying to phrase a wish properly. (And ironically, in this context, Eliezer asked me my background in computer science which essentially is asking what degree I have. I wonder if that was his true rejection.)
I think the original true rejection post is one of these. “Why should I believe him when he doesn’t have a PhD” probably means that the PhD is one of multiple conditions, not that it’s the sole reason for rejection. But Eliezer’s original post treats it as if it’s a gotcha.
Maybe, though if it’s the only condition given and I actually went out, got a PhD, came back, and the other person didn’t change their mind I’d be kind of annoyed. I have some direct experience with
if the potential customer says, “It seems good, but you don’t have feature X,” that may not be the true rejection. Fixing it may, or may not, change anything.
and anecdotal evidence seems in favour of Eliezer’s point. Having feature X doesn’t mean they’ll buy, and they might buy even without feature X if you change some other thing.
I’m noticing I’m not clear if we’re disagreeing here, or where that disagreement might lie. I think but am not at all confident that you think the unhelpful munchkinry is so prevalent and multiple rejections are sufficiently common as to make this technique useless. Can you try restate your point? At present this thread doesn’t seem productive to me.
I think but am not at all confident that you think the unhelpful munchkinry is so prevalent and multiple rejections are sufficiently common as to make this technique useless.
Pretty much.
If “true rejection” allows multiple reasons for rejection, not just one, and isn’t used as a gotcha, I have no real problem with it, but the difficulty of genie-proofing it and the ease of using it as a gotcha make it not work very well in practice.
This is also one reason I oppose bets.
I have some direct experience with
if the potential customer says, “It seems good, but you don’t have feature X,” that may not be the true rejection. Fixing it may, or may not, change anything.
But that sounds like the multiple reason case. If the customer has multiple reasons why he doesn’t like the product, and he’s only stated the biggest one, the customer may be unsatisfied after you fix it. Claiming “the customer didn’t state his true rejection” assumes that a true rejection can’t consist of multiple reasons. And even if the customer did phrase his rejection in some way that implies that it’s the only reason, I don’t think “failed to state all his reasons” is meaningfully like “didn’t give his true rejection”.
“Come back when your scientific journal is much later than 1880” is not particularly distinguishable, no.
Multiple things that I think are simultaneously true:
Something can satisfy all my pre-stated rejections and still not actually be good evidence for changing my mind. If that happens the right move is not to change my mind, because I don’t have good evidence.
There is a skill (or a cluster of related skills) which Is That Your True Rejection points at. Mastery of this skill is the ability to consider a belief of my own, concisely state the evidence which would change my mind on that belief, and be correct such that if that evidence is presented then my mind changes.
“I don’t like working out because I hate people watching me” is loosely isomorphic here to “I believe gravity on Earth accelerates objects down at about 9.8 m/s^2 because I dropped some objects off a high place and timed them.” The skill is pretty similar.
If someone points something out that satisfies my stated rejections, and I don’t change my mind, it’s pretty reasonable for them to be annoyed about that. I’m the one who failed a skill check.
If this keeps happening, where every time I try to pre-state my rejections and someone points out a way to satisfy them I find that I’m not convinced and add another reason, then I am not very good at the skill which Is That Your True Rejection points at.
There’s an unhelpful kind of munchkinry that looks for poor phrasing in pre-stated rejections like a devil in a fantasy novel or a highly motivated and perhaps unscrupulous lawyer, making suggestions that technically satisfy my rejections but obviously aren’t what I meant. There’s also a helpful kind of munchkinry that looks for out-of-the-box ways to work out without tripping anything we hate. The line is kind of blurry and may just outright be a gradient. I can’t write out rules that unhelpful munchkins couldn’t get around, but sometimes I know it when you see it and I don’t want to discourage helpful munchkinry.
Since the True Rejection Challenge exercise is in large part designed to practice the skill which Is That Your True Rejection points at, it’s going to involve a bunch of “oops, I didn’t think to add this rejection” in the beginning as well as the harder to spot failure of “oops, I added this as a rejection but it isn’t actually important to me.” When you start practicing juggling, you drop a lot of balls. That’s fine! Dropping balls in juggling practice and singing off key in choir practice aren’t mortal sins, they’re kind of supposed to happen! Obviously try not to make mistakes but go ahead and make lots of mistakes if it’s on the path to getting good at the skill.
In the case of prayer regrowing a leg, I think it’s possible someone failed a skill check by leaving the word “reputable” under-specified, and also possible their interlocutor is being an unhelpful munchkin. Both could be true at once! The helpful thing might be to say “hey, what do you mean by reputable?” Does that make sense?
I think the original true rejection post is one of these. “Why should I believe him when he doesn’t have a PhD” probably means that the PhD is one of multiple conditions, not that it’s the sole reason for rejection. But Eliezer’s original post treats it as if it’s a gotcha.
Also, remember the Hidden Complexity of Wishes. Phrasing your rejection in such a way that nobody can abuse a misphrasing is really hard, even with feedback, and a lot like trying to phrase a wish properly. (And ironically, in this context, Eliezer asked me my background in computer science which essentially is asking what degree I have. I wonder if that was his true rejection.)
Maybe, though if it’s the only condition given and I actually went out, got a PhD, came back, and the other person didn’t change their mind I’d be kind of annoyed. I have some direct experience with
and anecdotal evidence seems in favour of Eliezer’s point. Having feature X doesn’t mean they’ll buy, and they might buy even without feature X if you change some other thing.
I’m noticing I’m not clear if we’re disagreeing here, or where that disagreement might lie. I think but am not at all confident that you think the unhelpful munchkinry is so prevalent and multiple rejections are sufficiently common as to make this technique useless. Can you try restate your point? At present this thread doesn’t seem productive to me.
Pretty much.
If “true rejection” allows multiple reasons for rejection, not just one, and isn’t used as a gotcha, I have no real problem with it, but the difficulty of genie-proofing it and the ease of using it as a gotcha make it not work very well in practice.
This is also one reason I oppose bets.
But that sounds like the multiple reason case. If the customer has multiple reasons why he doesn’t like the product, and he’s only stated the biggest one, the customer may be unsatisfied after you fix it. Claiming “the customer didn’t state his true rejection” assumes that a true rejection can’t consist of multiple reasons. And even if the customer did phrase his rejection in some way that implies that it’s the only reason, I don’t think “failed to state all his reasons” is meaningfully like “didn’t give his true rejection”.