I was one of the people putting down “40%” or “60%” type answers—despite also thinking “man, I’m really not sure here.” But “40%/60%” feels like a number that’s “sufficiently” uncertain to represent a rough mental state of “not really sure, but if I had to pick I’d go with meat-eater.” When it fact if anything I should have been putting down 49%/51% at best, because seriously it was genuinely hard to tell.
I also remember the essays being very weird and throwing me off a bit. Like, in the normal population, vegetarians say “animals can feel pain and we shouldn’t hurt or kill them” and meat eaters say “we’re at the top of the food chain and it’s the natural order of things for us to eat.”
On less wrong, vegetarians say things like “animals feel pain, therefore factory farming is evil but also we probably should destroy all natural habitat so that wild animals can’t exist or suffer. Also maybe we should just eradicate all life on earth to prevent suffering.” And meat eaters say “I look forward to tasty cost effective meat grown in test tubes but right now I’m too lazy and don’t care that much. Also, probably better to have slightly-better farming conditions but then continue creating billions of chickens for a net hedonic gain.”
It was pretty clear that any lessons I took on identifying vegetarians/meat-eaters from this test were pretty localized.
I suspect that at least some judges (including me, though I’m reconstructing rather than actually recalling my thought processes) (1) used 40⁄60 to indicate “meh, scarcely any idea but I lean this way rather than that” and then (2) felt like they had to use 30⁄70 for opinions one notch stronger, even though if evaluating them in a vacuum they might have chosen something more like 40% or 60% to represent them.
(In my case, at least, this doesn’t make me look much better; even aside from the fact that that isn’t how you should assign probabilities, I got exactly half of my eight 40⁄60 judgements right, and also exactly half of my four 30⁄70 judgements. I suppose that means I’m consistent, but not in a good way.)
In retrospect I ought to have included options closer to 50%. I didn’t expect that they’d be so necessary! You are absolutely right, though.
A big part of LessWrong, I think, is learning to overcome our mental failings. Perhaps we can use this as a lesson that the best judge writes down their credence before seeing the options, then picks the option that is the best match to what they wrote. I know that I, personally, try (and often fail) to use this technique when doing multiple-choice tests.
I do a similar thing when contemplating buying things like books: I don’t look at the price, then I ask myself “How much would I pay to have this?”, then I check the price. (And then, if it’s a book, I too often buy the damn thing anyway even though the price is higher than the one I decided on. Perfectly rational action is, alas, beyond me.)
Looking at the price gives you information about how valuable and/or rare the thing is, which may in turn affect what price you are willing to pay in such a way that cannot be captured by guessing a price beforehand.
Of course this can be gamed by sellers in specific situations, but sellers who did that every single time would gain a reputation for low-information pricing, which would limit how many sellers actually do that.
Yes, but in the cases I have in mind (e.g., buying a book) it gives very little extra information. If I’ve already had a good look inside a book to assess how interesting the subject matter is, how clearly things are explained, how well written the prose is, how competent the typography is, what sort of paper it’s printed on, etc., knowing the price will tell me basically nothing more about how valuable the book will be to me.
I would not encourage this technique for goods whose present and future price are largely driven by scarcity and whose value to you is substantially affected by your prospects of selling them in the future. But if you’re buying such goods, you should be getting scarcity information from other places besides the current asking price of the particular instance now before you.
Interesting update/realization I just had:
I was one of the people putting down “40%” or “60%” type answers—despite also thinking “man, I’m really not sure here.” But “40%/60%” feels like a number that’s “sufficiently” uncertain to represent a rough mental state of “not really sure, but if I had to pick I’d go with meat-eater.” When it fact if anything I should have been putting down 49%/51% at best, because seriously it was genuinely hard to tell.
I also remember the essays being very weird and throwing me off a bit. Like, in the normal population, vegetarians say “animals can feel pain and we shouldn’t hurt or kill them” and meat eaters say “we’re at the top of the food chain and it’s the natural order of things for us to eat.”
On less wrong, vegetarians say things like “animals feel pain, therefore factory farming is evil but also we probably should destroy all natural habitat so that wild animals can’t exist or suffer. Also maybe we should just eradicate all life on earth to prevent suffering.” And meat eaters say “I look forward to tasty cost effective meat grown in test tubes but right now I’m too lazy and don’t care that much. Also, probably better to have slightly-better farming conditions but then continue creating billions of chickens for a net hedonic gain.”
It was pretty clear that any lessons I took on identifying vegetarians/meat-eaters from this test were pretty localized.
But we didn’t have that option!
I suspect that at least some judges (including me, though I’m reconstructing rather than actually recalling my thought processes) (1) used 40⁄60 to indicate “meh, scarcely any idea but I lean this way rather than that” and then (2) felt like they had to use 30⁄70 for opinions one notch stronger, even though if evaluating them in a vacuum they might have chosen something more like 40% or 60% to represent them.
(In my case, at least, this doesn’t make me look much better; even aside from the fact that that isn’t how you should assign probabilities, I got exactly half of my eight 40⁄60 judgements right, and also exactly half of my four 30⁄70 judgements. I suppose that means I’m consistent, but not in a good way.)
In retrospect I ought to have included options closer to 50%. I didn’t expect that they’d be so necessary! You are absolutely right, though.
A big part of LessWrong, I think, is learning to overcome our mental failings. Perhaps we can use this as a lesson that the best judge writes down their credence before seeing the options, then picks the option that is the best match to what they wrote. I know that I, personally, try (and often fail) to use this technique when doing multiple-choice tests.
Yup.
I do a similar thing when contemplating buying things like books: I don’t look at the price, then I ask myself “How much would I pay to have this?”, then I check the price. (And then, if it’s a book, I too often buy the damn thing anyway even though the price is higher than the one I decided on. Perfectly rational action is, alas, beyond me.)
Looking at the price gives you information about how valuable and/or rare the thing is, which may in turn affect what price you are willing to pay in such a way that cannot be captured by guessing a price beforehand.
Of course this can be gamed by sellers in specific situations, but sellers who did that every single time would gain a reputation for low-information pricing, which would limit how many sellers actually do that.
Yes, but in the cases I have in mind (e.g., buying a book) it gives very little extra information. If I’ve already had a good look inside a book to assess how interesting the subject matter is, how clearly things are explained, how well written the prose is, how competent the typography is, what sort of paper it’s printed on, etc., knowing the price will tell me basically nothing more about how valuable the book will be to me.
I would not encourage this technique for goods whose present and future price are largely driven by scarcity and whose value to you is substantially affected by your prospects of selling them in the future. But if you’re buying such goods, you should be getting scarcity information from other places besides the current asking price of the particular instance now before you.