It is good to taboo words, but it is also good to criticize the attempts of others to taboo words, if you can make the case that those attempts fail to capture something important.
For example, it seems possible that a computer could predict your actions to high precision, but by running computations so different from the ones that you would have run yourself that the simulated-you doesn’t have subjective experiences. (If I understand it correctly, this is the idea behind Eliezer’s search for a non-person predicate. It would be good if this is possible, because then a superintelligence could run alternate histories without torturing millions of sentient simulated beings.) If such a thing is possible, then any superficial behavioristic attempt to taboo “subjective experience” will be missing something important.
Furthermore, I can mount this critique of such an attempt without being obliged to taboo “subjective experience” myself. That is, making the critique is valuable even if it doesn’t offer an alternative way to taboo “subjective experience”.
It’s not clear to me that “understanding” means “subjective experience,” which is one of several reasons why I think it’s reasonable for me to ask that we taboo “understanding.”
It is good to taboo words, but it is also good to criticize the attempts of others to taboo words, if you can make the case that those attempts fail to capture something important.
For example, it seems possible that a computer could predict your actions to high precision, but by running computations so different from the ones that you would have run yourself that the simulated-you doesn’t have subjective experiences. (If I understand it correctly, this is the idea behind Eliezer’s search for a non-person predicate. It would be good if this is possible, because then a superintelligence could run alternate histories without torturing millions of sentient simulated beings.) If such a thing is possible, then any superficial behavioristic attempt to taboo “subjective experience” will be missing something important.
Furthermore, I can mount this critique of such an attempt without being obliged to taboo “subjective experience” myself. That is, making the critique is valuable even if it doesn’t offer an alternative way to taboo “subjective experience”.
It’s not clear to me that “understanding” means “subjective experience,” which is one of several reasons why I think it’s reasonable for me to ask that we taboo “understanding.”
I didn’t mean to suggest that “understanding” means “subjective experience”, or to suggest that anyone else was suggesting that.