To measure nonconformity I needed to define “nonconformity”.
I think this is a fairly big mistake.
First, as a practical matter, one does not actually need to define something in order to measure it; often, the process works in the reverse order. For instance, I would guess that early scientists trying to measure temperature or air pressure first made a measurement device, then defined “temperature” or “air pressure” as the thing they measured. (Or if they did try to define temperature/air pressure beforehand, their definitions were probably incomplete/wrong until after they had the measurement devices.)
Second, and more importantly: when someone wants to “define” a word, they are usually confused about how words work. Definitions, as we usually use them, are not the correct data structure for word-meaning. Words point to clusters in thing-space; definitions try to carve up those clusters with something like cutting-planes. That’s an unreliable and very lossy way to represent clusters, and can’t handle edge-cases well or ambiguous cases at all.
If you want to measure some abstract thing like “nonconformity”, then I’d suggest a process more like this:
Come up with a bunch of crappy proxy measures.
Go measure them all a bunch in the real world.
Do a factor analysis and see if there’s one big dominant factor which intuitively seems to match “nonconformity”.
Note that you may not find one big factor which intuitively seems to match “nonconformity”! Whether “nonconformity” is a useful, predictive abstraction at all is an empirical question.
(Side note: I jumped from talking-about-clusters to talking-about-factor-analysis. Mathematically, these both do a very similar thing, at least if we’re talking about Bayesian clustering models: both try to find some relatively-low-dimensional latent variables such that our observables are conditionally independent given the latents.)
Alternatively, rather than a formal factor analysis, you could use the intuitive equivalent: take a bunch of “proxy measures” or even just a list of examples, and try to intuit the unifying, shared aspect which makes them all good proxies/examples. This can be “less noisy” than a formal factor analysis, since you have some intuition for which-parts-are-important. This is especially useful for coming up with good mathematical definitions.
I think this is a fairly big mistake.
First, as a practical matter, one does not actually need to define something in order to measure it; often, the process works in the reverse order. For instance, I would guess that early scientists trying to measure temperature or air pressure first made a measurement device, then defined “temperature” or “air pressure” as the thing they measured. (Or if they did try to define temperature/air pressure beforehand, their definitions were probably incomplete/wrong until after they had the measurement devices.)
Second, and more importantly: when someone wants to “define” a word, they are usually confused about how words work. Definitions, as we usually use them, are not the correct data structure for word-meaning. Words point to clusters in thing-space; definitions try to carve up those clusters with something like cutting-planes. That’s an unreliable and very lossy way to represent clusters, and can’t handle edge-cases well or ambiguous cases at all.
If you want to measure some abstract thing like “nonconformity”, then I’d suggest a process more like this:
Come up with a bunch of crappy proxy measures.
Go measure them all a bunch in the real world.
Do a factor analysis and see if there’s one big dominant factor which intuitively seems to match “nonconformity”.
Note that you may not find one big factor which intuitively seems to match “nonconformity”! Whether “nonconformity” is a useful, predictive abstraction at all is an empirical question.
(Side note: I jumped from talking-about-clusters to talking-about-factor-analysis. Mathematically, these both do a very similar thing, at least if we’re talking about Bayesian clustering models: both try to find some relatively-low-dimensional latent variables such that our observables are conditionally independent given the latents.)
Alternatively, rather than a formal factor analysis, you could use the intuitive equivalent: take a bunch of “proxy measures” or even just a list of examples, and try to intuit the unifying, shared aspect which makes them all good proxies/examples. This can be “less noisy” than a formal factor analysis, since you have some intuition for which-parts-are-important. This is especially useful for coming up with good mathematical definitions.