Currently, my first-pass check for “is this probably a natural abstraction?” is “can humans usually figure out what I’m talking about from a few examples, without a formal definition?”. For human values, the answer seems like an obvious “yes”. For evolutionary fitness… nonobvious. Humans usually get it wrong without the formal definition.
Hmm, presumably you’re not including something like “internal consistency” in the definition of ‘natural abstraction’. That is, humans who aren’t thinking carefully about something will think there’s an imaginable object even if any attempts to actually construct that object will definitely lead to failure. (For example, Arrow’s Impossibility Theorem comes to mind; a voting rule that satisfies all of those desiderata feels like a ‘natural abstraction’ in the relevant sense, even though there aren’t actually any members of that abstraction.)
Oh this is fascinating. This is basically correct; a high-level model space can include models which do not correspond to any possible low-level model.
One caveat: any high-level data or observations will be consistent with the true low-level model. So while there may be natural abstract objects which can’t exist, and we can talk about those objects, we shouldn’t see data supporting their existence—e.g. we shouldn’t see a real-world voting system behaving like it satisfies all of Arrow’s desiderata.
Hmm, presumably you’re not including something like “internal consistency” in the definition of ‘natural abstraction’. That is, humans who aren’t thinking carefully about something will think there’s an imaginable object even if any attempts to actually construct that object will definitely lead to failure. (For example, Arrow’s Impossibility Theorem comes to mind; a voting rule that satisfies all of those desiderata feels like a ‘natural abstraction’ in the relevant sense, even though there aren’t actually any members of that abstraction.)
Oh this is fascinating. This is basically correct; a high-level model space can include models which do not correspond to any possible low-level model.
One caveat: any high-level data or observations will be consistent with the true low-level model. So while there may be natural abstract objects which can’t exist, and we can talk about those objects, we shouldn’t see data supporting their existence—e.g. we shouldn’t see a real-world voting system behaving like it satisfies all of Arrow’s desiderata.