Two categories that don’t quite match the ones you laid out here.
I think there is something like “being a good citizen when trying to create jargon.” Don’t pick a word that everyone will predictably misunderstand, or will predictably really want to use for some other more common thing, if you want to also be able to have conversations with that “everyone.”
This isn’t (primarily) about fighting political/hype cycles, it’s just… like, well one negative example I updated on: Eliezer defines meta-honesty to be “be at least as honest as a highly honest person AND ALSO always be honest about under what circumstances you will be honest.” He tacked on the first part for a reason (to avoid accidentally encouraging people to use “metahonesty” for clever self-serving arguments). But, frankly, “metahonesty” is a pretty self-explanatory word if it just means the second thing, and most people will probably interpret it to mean just the “be honest about being honest” part.
I think the bundle-of-concepts Eliezer wanted to point to should be called something more like “Eliezer’s Code of (Meta)-honesty” or something catchier but more oddly specific. And let “metahonesty” just be a technical term that isn’t also trying to be a code of honor, that means what it sounds like it should mean.
...
Also, re: staying ahead of a political race. It’s kinda reasonable to just Not Wanna Play That Game, but, note that a lot of the stakes here is not “doing politics Out There somewhere”, it’s having terminology that keeps making sense in the intellectual circles. If most of the people studying AI, even from perspective of AI safety, end up studing “weak superintelligences”, trying to preserve a definition that uses it to always mean “overwhelmingly strong” is setting yourself up for a lot of annoying conversations just while trying to discuss concepts intellectually.
Maybe with meta-honesty it should be solved something like that meta-honesty means what it seems to mean, but to save the same thing that EY wanted, you have a remark that meta-honesty doesn’t negate simple honesty, you don’t get to say “ah, but I was meta-honest in the sense that I didn’t tell anyone that what I say is truth, therefore they shouldn’t have expected it as such”. It seems to me to perform the same role without adding some unobvuous meaning into the term. Sort of, relying on the same thing as that if you say how you are able to take out someone’s mother from burning house, that doesn’t mean that you can just do it as a corpse, your claim about ability here should mean something additional, having all else equal. And same for meta honesty, saying you are honest meta you don’t get to use some language of your own invention where yes means no or anything like that. This seems as intuitive way to put it, keeping both the idea that you don’t get to lie by simply not claiming your words to be true and having the term definition not be a complicated unexpected bundle.
Two categories that don’t quite match the ones you laid out here.
I think there is something like “being a good citizen when trying to create jargon.” Don’t pick a word that everyone will predictably misunderstand, or will predictably really want to use for some other more common thing, if you want to also be able to have conversations with that “everyone.”
This isn’t (primarily) about fighting political/hype cycles, it’s just… like, well one negative example I updated on: Eliezer defines meta-honesty to be “be at least as honest as a highly honest person AND ALSO always be honest about under what circumstances you will be honest.” He tacked on the first part for a reason (to avoid accidentally encouraging people to use “metahonesty” for clever self-serving arguments). But, frankly, “metahonesty” is a pretty self-explanatory word if it just means the second thing, and most people will probably interpret it to mean just the “be honest about being honest” part.
I think the bundle-of-concepts Eliezer wanted to point to should be called something more like “Eliezer’s Code of (Meta)-honesty” or something catchier but more oddly specific. And let “metahonesty” just be a technical term that isn’t also trying to be a code of honor, that means what it sounds like it should mean.
...
Also, re: staying ahead of a political race. It’s kinda reasonable to just Not Wanna Play That Game, but, note that a lot of the stakes here is not “doing politics Out There somewhere”, it’s having terminology that keeps making sense in the intellectual circles. If most of the people studying AI, even from perspective of AI safety, end up studing “weak superintelligences”, trying to preserve a definition that uses it to always mean “overwhelmingly strong” is setting yourself up for a lot of annoying conversations just while trying to discuss concepts intellectually.
Maybe with meta-honesty it should be solved something like that meta-honesty means what it seems to mean, but to save the same thing that EY wanted, you have a remark that meta-honesty doesn’t negate simple honesty, you don’t get to say “ah, but I was meta-honest in the sense that I didn’t tell anyone that what I say is truth, therefore they shouldn’t have expected it as such”. It seems to me to perform the same role without adding some unobvuous meaning into the term. Sort of, relying on the same thing as that if you say how you are able to take out someone’s mother from burning house, that doesn’t mean that you can just do it as a corpse, your claim about ability here should mean something additional, having all else equal. And same for meta honesty, saying you are honest meta you don’t get to use some language of your own invention where yes means no or anything like that. This seems as intuitive way to put it, keeping both the idea that you don’t get to lie by simply not claiming your words to be true and having the term definition not be a complicated unexpected bundle.