I think there’s something to be said for this straightforward counterargument here: conceptual rounding off is performing the valuable service of lumping together ideas that are not that different and only seem importantly different to their authors/supporters rather than actually meaningfully different ideas.
An idea should either be precisely defined enough that it’s clear why it can’t be rounded off (once the precise definition is known), or it’s a vague idea and it either needs to become more precise to avoid being rounded or it is inherently vague and being vague there can’t be much harm from rounding because it already wasn’t clear where its boundaries were in concept space.
So… there surely are things like (overlapping, likely non-exhaustive):
Memetic Darwinian anarchy—concepts proliferating without control, trying to carve out for themselves new niches in the noosphere or grab parts of real estate belonging to incumbent concepts.
Memetic warfare—individuals, groups, egregores, trying to control the narrative by describing the same thing in the language of your own ideology, yadda yadda.
Independent invention of the same idea—in which case it’s usually given different names (but also, plausibly, since some people may grow attached to their concepts of choice, they might latch onto trivial/superficial differences and amplify that, so that one or more instances of this multiply independently invented concept now is now morphed into something else than what it “should be”).
Memetic rent seeking—because introducing a new catchy concept might marginally bump up your h-index.
Still, the thing Jan describes is real and often a big problem.
I also think I somewhat disagree with this:
An idea should either be precisely defined enough that it’s clear why it can’t be rounded off (once the precise definition is known), or it’s a vague idea and it either needs to become more precise to avoid being rounded or it is inherently vague and being vague there can’t be much harm from rounding because it already wasn’t clear where its boundaries were in concept space.
Meanings are often subtle, intuited but not fully grasped, in which case a (premature) attempt to explicitize them risks collapsing their reference to the important thing they are pointing at. Many important concepts are not precisely defined. Many are best sorta-defined ostensively: “examples of X include A, B, C, D, and E; I’m not sure what it makes all of them instances of X, maybe it’s that they share the properties Y and Z … or at least my best guess is that Y and Z are important parts of X and I’m pretty sure that X is a Thing™”.
Eliezer has a post (I couldn’t find it at the moment) where he noticed that the probabilities he gave were inconsistent. He asks something like, “Would I really not behave as if God existed if I believed that P(Christianity)=1e-5?” and then, “Oh well, too bad, but I don’t know which way to fix it, and fixing it either way risks losing important information, so I’m deciding to live with this lack of consistency for now.”
The seemingly small differences might matter hugely. See the long debate over what caused scurvy and how to prevent/cure it.
When the Royal Navy changed from using Sicilian lemons to West Indian limes, cases of scurvy reappeared. The limes were thought to be more acidic and it was therefore assumed that they would be more effective at treating scurvy. However, limes actually contain much less vitamin C and were consequently much less effective.
Furthermore, fresh fruit was substituted with lime juice that had often been exposed to either air or copper piping. This resulted in at least a partial removal of vitamin C from the juice, thus reducing its effectiveness.
The discovery that fresh meat was able to cure scurvy was another reason why people no longer treated the condition with fresh fruit. This discovery led to the belief that perhaps scurvy was not caused by a dietary problem at all. Instead, it was thought to be the result of a bacterial infection from tainted meat. In fact, the healing properties of fresh meat come from the high levels of vitamin C it contains.
Finally, the arrival of steam shipping substantially reduced the amount of time people spent at sea, therefore the difficulties in carrying enough fresh produce were reduced. This decreased the risk of scurvy so that less effective treatments, such as lime juice, proved effective enough to deal with the condition most of the time.
TBC the scurvy story was very complicated. Im not actually sure the above summary covers it completely accurately. Though it illustrates some of the factors. Conceptual lumping was a real impediment to figuring out was going on!
...The loss of knowledge has been attributed to several factors. Firstly, Lind showed in his work that there was no connection between the acidity of the citrus fruit and its effectiveness at curing scurvy. In particular, he noted that acids alone (sulphuric acid or vinegar), would not suffice. Despite this, it remained a popular theory that any acid could be used in place of citrus fruit. This misconception had significant consequences.
When the Royal Navy changed from using Sicilian lemons to West Indian limes, cases of scurvy reappeared. The limes were thought to be more acidic and it was therefore assumed that they would be more effective at treating scurvy. However, limes actually contain much less vitamin C and were consequently much less effective. Furthermore, fresh fruit was substituted with lime juice that had often been exposed to either air or copper piping. This resulted in at least a partial removal of vitamin C from the juice, thus reducing its effectiveness.
The discovery that fresh meat was able to cure scurvy was another reason why people no longer treated the condition with fresh fruit. This discovery led to the belief that perhaps scurvy was not caused by a dietary problem at all. Instead, it was thought to be the result of a bacterial infection from tainted meat. In fact, the healing properties of fresh meat come from the high levels of vitamin C it contains.
Finally, the arrival of steam shipping substantially reduced the amount of time people spent at sea, therefore the difficulties in carrying enough fresh produce were reduced. This decreased the risk of scurvy so that less effective treatments, such as lime juice, proved effective enough to deal with the condition most of the time. Unfortunately, this meant that knowledge of the most effective treatment for scurvy was gradually lost....
I think there’s something to be said for this straightforward counterargument here: conceptual rounding off is performing the valuable service of lumping together ideas that are not that different and only seem importantly different to their authors/supporters rather than actually meaningfully different ideas.
An idea should either be precisely defined enough that it’s clear why it can’t be rounded off (once the precise definition is known), or it’s a vague idea and it either needs to become more precise to avoid being rounded or it is inherently vague and being vague there can’t be much harm from rounding because it already wasn’t clear where its boundaries were in concept space.
So… there surely are things like (overlapping, likely non-exhaustive):
Memetic Darwinian anarchy—concepts proliferating without control, trying to carve out for themselves new niches in the noosphere or grab parts of real estate belonging to incumbent concepts.
Memetic warfare—individuals, groups, egregores, trying to control the narrative by describing the same thing in the language of your own ideology, yadda yadda.
Independent invention of the same idea—in which case it’s usually given different names (but also, plausibly, since some people may grow attached to their concepts of choice, they might latch onto trivial/superficial differences and amplify that, so that one or more instances of this multiply independently invented concept now is now morphed into something else than what it “should be”).
Memetic rent seeking—because introducing a new catchy concept might marginally bump up your h-index.
So, as usual, the law of equal and opposite advice applies.
Still, the thing Jan describes is real and often a big problem.
I also think I somewhat disagree with this:
Meanings are often subtle, intuited but not fully grasped, in which case a (premature) attempt to explicitize them risks collapsing their reference to the important thing they are pointing at. Many important concepts are not precisely defined. Many are best sorta-defined ostensively: “examples of X include A, B, C, D, and E; I’m not sure what it makes all of them instances of X, maybe it’s that they share the properties Y and Z … or at least my best guess is that Y and Z are important parts of X and I’m pretty sure that X is a Thing™”.
Eliezer has a post (I couldn’t find it at the moment) where he noticed that the probabilities he gave were inconsistent. He asks something like, “Would I really not behave as if God existed if I believed that P(Christianity)=1e-5?” and then, “Oh well, too bad, but I don’t know which way to fix it, and fixing it either way risks losing important information, so I’m deciding to live with this lack of consistency for now.”
The seemingly small differences might matter hugely. See the long debate over what caused scurvy and how to prevent/cure it.
TBC the scurvy story was very complicated. Im not actually sure the above summary covers it completely accurately. Though it illustrates some of the factors. Conceptual lumping was a real impediment to figuring out was going on!
Link to the source of the quote?
A google of the first paragraph takes you quickly to https://www.bluesci.co.uk/posts/forgotten-knowledge
Which paragraph exactly? I tried and this did not replicate
Ah thanks