The thesis of this post is that people who think neural fireplaces are fireplaces should be arguing for that on the merits—that the decision-relevant thing is having the subjective experience of a fireplace, even if the hallucinations don’t provide heat or light. They shouldn’t be saying, “We prefer to draw our categories this way because otherwise the CEO of Neural Fireplaces, Inc. will be really sad, and he’s our friend.”
Hmm, what is the difference between these two types of arguments? I could recast the latter argument in terms of “features to pay attention to” or “what’s decision relevant”: If we pay attention to the feature of “things that the CEO of Neural Fireplaces wants people to treat as interchangeable to the greatest extent possible” then electric, fuel-burning, and neural fireplaces form a natural cluster. The CEO is our friend so it’s highly decision relevant to consider what things he wants us to treat as interchangeable.
In the OP you talk about how redrawing categories for non-epistemic reasons would interfere with Bayesian reasoning, but that applies the former type of argument as well. If we decide to include neural fireplaces in the “fireplace” category based on that the decision-relevant thing is having the subjective experience of a fireplace, it equally interferes with Bayesian reasoning: we can no longer safely infer “generates objective light with high probability” upon hearing “fireplace”, and some people may well make erroneous inferences during a transition period before everyone got on the same page.
So hopefully I’m not being willfully obtuse, but I’m not sure what principle you’re drawing on to say that the former type of argument is ok but the latter is not.
But presumably the reason the CEO would be sad if people didn’t consider neural fireplaces to be fireplaces is because he wants to be leading a successful company that makes things people want, not a useless company with a useless product. Redefining words “in the map” doesn’t help achieve goals “in the territory”.
The OP discusses a similar example about wanting to be funny. If I think I can get away with changing the definition of the word “funny” such that it includes my jokes by definition, I’m less likely to try interventions that will make people want to watch my stand-up routine, which is one of the consequences I care about that the old concept of funny pointed to and the new concept doesn’t.
Now, it’s true that, in all metaphysical strictness, the map is part of the territory. “what the CEO thinks” and “what we’ve all agreed to put in the same category” are real-world criteria that one can use to discriminate between entities.
But if you’re not trying to deceive someone by leveraging ambiguity between new and old definitions, it’s hard to see why someone would care about such “thin” categories (simply defined by fiat, rather than pointing to a cluster in a “thicker”, higher-dimensional subspace of related properties). The previous post discusses the example of a “Vice President” job title that’s identical to a menial job in all but the title itself: if being a “Vice President” doesn’t imply anything about pay or authority or job duties, it’s not clear why I would particularly want to be a “Vice President”, except insofar as I’m being fooled by what the term used to mean.
But presumably the reason the CEO would be sad if people didn’t consider neural fireplaces to be fireplaces is because he wants to be leading a successful company that makes things people want, not a useless company with a useless product. Redefining words “in the map” doesn’t help achieve goals “in the territory”.
I see, I think this makes sense, but it depends on the CEO’s actual goals/values, right? What if the CEO wants to leverage his friendships to make money, and doesn’t mind people buying neural fireplaces partly or wholly out of care/sympathy for him? And everyone is (or most people are) happy to do this out of genuine care/sympathy for the CEO? In that case there is seemingly no deception involved, and redefining words “in the map” does help achieve goals “in the territory”.
Which of these two analogies is closer to the transgender situation involves empirical questions that I lack the knowledge to discuss. But it occurs to me that maybe your disagreement with Eliezer/Scott is based on you thinking that the first analogy is closer, and them thinking that the second analogy is closer? In other words, maybe they think that trans people would be happy enough with people treating them as their preferred sex/gender out of care/sympathy, and not necessarily “on the merits” in some way?
Hmm, what is the difference between these two types of arguments? I could recast the latter argument in terms of “features to pay attention to” or “what’s decision relevant”: If we pay attention to the feature of “things that the CEO of Neural Fireplaces wants people to treat as interchangeable to the greatest extent possible” then electric, fuel-burning, and neural fireplaces form a natural cluster. The CEO is our friend so it’s highly decision relevant to consider what things he wants us to treat as interchangeable.
In the OP you talk about how redrawing categories for non-epistemic reasons would interfere with Bayesian reasoning, but that applies the former type of argument as well. If we decide to include neural fireplaces in the “fireplace” category based on that the decision-relevant thing is having the subjective experience of a fireplace, it equally interferes with Bayesian reasoning: we can no longer safely infer “generates objective light with high probability” upon hearing “fireplace”, and some people may well make erroneous inferences during a transition period before everyone got on the same page.
So hopefully I’m not being willfully obtuse, but I’m not sure what principle you’re drawing on to say that the former type of argument is ok but the latter is not.
But presumably the reason the CEO would be sad if people didn’t consider neural fireplaces to be fireplaces is because he wants to be leading a successful company that makes things people want, not a useless company with a useless product. Redefining words “in the map” doesn’t help achieve goals “in the territory”.
The OP discusses a similar example about wanting to be funny. If I think I can get away with changing the definition of the word “funny” such that it includes my jokes by definition, I’m less likely to try interventions that will make people want to watch my stand-up routine, which is one of the consequences I care about that the old concept of funny pointed to and the new concept doesn’t.
Now, it’s true that, in all metaphysical strictness, the map is part of the territory. “what the CEO thinks” and “what we’ve all agreed to put in the same category” are real-world criteria that one can use to discriminate between entities.
But if you’re not trying to deceive someone by leveraging ambiguity between new and old definitions, it’s hard to see why someone would care about such “thin” categories (simply defined by fiat, rather than pointing to a cluster in a “thicker”, higher-dimensional subspace of related properties). The previous post discusses the example of a “Vice President” job title that’s identical to a menial job in all but the title itself: if being a “Vice President” doesn’t imply anything about pay or authority or job duties, it’s not clear why I would particularly want to be a “Vice President”, except insofar as I’m being fooled by what the term used to mean.
I see, I think this makes sense, but it depends on the CEO’s actual goals/values, right? What if the CEO wants to leverage his friendships to make money, and doesn’t mind people buying neural fireplaces partly or wholly out of care/sympathy for him? And everyone is (or most people are) happy to do this out of genuine care/sympathy for the CEO? In that case there is seemingly no deception involved, and redefining words “in the map” does help achieve goals “in the territory”.
Which of these two analogies is closer to the transgender situation involves empirical questions that I lack the knowledge to discuss. But it occurs to me that maybe your disagreement with Eliezer/Scott is based on you thinking that the first analogy is closer, and them thinking that the second analogy is closer? In other words, maybe they think that trans people would be happy enough with people treating them as their preferred sex/gender out of care/sympathy, and not necessarily “on the merits” in some way?