I think there may be more truth in what you’re saying than a lot of people would be entirely comfortable admitting… although what’s a cause and what’s an effect is hard to say.
… but I also think that going around looking for “compact generators” of human behavior, especially social behavior, is basically asking to be wrong. In fact, you can apply a sort of anti-parsimony: if a theory claims to compactly generate any significant set of social dynamics, that’s evidence against the theory. People are constantly coming up with simple explanations, and they’re constantly turning out to be wrong, and any given simple explanation has to overcome that prior.
A whole lot of things are allowed to be going on at the same time, pushing toward the same or similar results, or more likely sometimes pushing in the same direction and at other times conflicting. There are allowed to be arbitrarily complicated networks of both positive and negative feedback. Which things are most important is allowed to change not just from person to person, but from time to time.
So, there’s this thing called Solomonoff induction. It works, provably, for anything Turing computable. And human social behavior is definitely Turing computable.
“If a theory claims to compactly generate any significant set of social dynamics, that’s evidence against the theory” is an anti-inductive prior. It’s like saying that things which have happened less often before are more likely in the future, and therefore the sun will certainly not rise tomorrow.
A thing in the neighbourhood which sorta looks like an anti-parsimony prior is an “anti-wishful-thinking prior”. That is, we might think that humans sometimes have a tendency to latch onto overly simply theories and discard some of the data they should be explaining, so when someone proposes a theory that seems overly simple relative to the expected complexity of the domain they’re trying to explain, it could be rational to update that they’re likely engaging in “wishful thinking”, when compared to someone with a more complex theory. e.g. if someone posts a proof of the Riemann Hypothesis that’s ten pages long, or a very simple analysis of a trend in the stock market purported to give huge gains, etc....
It’s not anti-inductive to believe that sets of phenomena in [some domain] are unlikely to have simple explanations at [some level of abstraction].
WRT social behavior, it does seem to me that ‘a whole lot of things pushing toward the same or similar results’ is remarkably common, and the OP is too dismissive of other explanations, once it’s thought of one, for my priors. (At the same time, I totally think [simple theories that help explain significant sets of dynamics] are real.)
As alluded to by the name of the website, part of Solomonoff/MDL is that there doesn’t necessarily have to be a unique “correct” explanation: theories are better to the extent that their predictions pay for their complexity. It’s not that compact generators are necessarily “true”; it’s that if a compact generator is yielding OK predictions, then more complex theories need to be making better predictions to single themselves out. You shouldn’t say that looking for compact generators of a complex phenomenon is asking to be wrong unless you have a way to be less wrong.
IME in this domain, in epistemology-for-humans terms that may or may not translate easily into Solomonoff/MDL, taking a compact generator of a complex phenomenon too seriously — like, concentrating probability too strongly on its predictions, not taking anomalies seriously enough or looking hard enough for them, insufficiently expecting there to be more to say that sounds different in kind — is asking to be wrong, and not doing that is a way to be less wrong.
(Looking for compact generators but not taking them too seriously is good, but empirically seems to require more skill or experience.)
You don’t need to single out a specific complex theory to say “this simple theory is concentrating probability too strongly”, or to expect there to be some complex theory that pays for itself.
Solomonoff induction gives you a weighted sum over an infinite number of programs[1]. That’s not compact. And if were computable, which it isn’t, or even approximable, which it probably isn’t for this case, I doubt you’d be able to collect enough data in your lifetime for it to converge to speak of. Not even assuming that you were able to reliably collect all relevant data, which you’re not, and that you were actually encoding or processing the data in a formal way, which you’re also not.
And if you actually did somehow get your hands around a Solomonoff sum, you still wouldn’t be able to just grab a single term out of it, not even the one for the shortest program, and substitute it as “the” explanation on the grounds that “Solomonoff induction works”.
I can understand “compact generation” as a metaphorical allusion to Occam, but seriously, Solomoff induction isn’t even useful as a metaphor for any well-chosen approach here. You can’t let formalisms like that invade your thinking to the point where you seriously think in terms of them in areas where it doesn’t make sense.
Also, human social behavior probably isn’t deterministically Turing computable even if you model the entire universe. Probabilistically computable, probably, yes. In theory. And to be fair I’m sure Solomonoff goes through just fine to nondeterministic Turing processes. But anyway, you don’t actually have, and can’t actually get, a machine that computes human behavior or even a meaningful approximation to it.
There’s also no anti-inductive prior involved. What I’m saying isn’t about the underlying phenomena at all, and certainly doesn’t say that there’s no regularity in them. It’s about the theory, and it has in fact happened, far more often than not in my experience, that simple, single-explanation, “compact” theories, yield really bogus results.
Which is actually capable of encoding “lots of different, interacting things are going on” in a way that a single, deterministic Turing program would not be.
i notice that it’s long been dogma that compact generators for human CEV are instantly and totally distrustworthy in exactly this way, that their compactness is very strong evidence against them
this feels related, but i’m not actually willing to stick my neck out and say it’s 1:1
I’d say that different people want/do different things (otherwise we probably wouldn’t have this debate right now; no one makes LW posts with convincing evolutionary arguments for why people breathe), therefore you should be suspicious of theories that predict that everyone wants the same thing.
You can get wronger faster by using complex generators than compact generators.
Matching the complexity of the hypothesis to expected complexity given the domain seems like an interesting idea.
“People and situations being different from one another so you should understand them carefully” seem like an important and true message, worth belabouring, even though it seems clear to me johnswentworth already knows and acknowledges this.
This comment feels to me like it might be dancing around saying “Hey! Don’t rape people! Make sure you are not raping people! You are saying some pretty rapey things!” which feels like a more useful message, but one that feels more unpleasant to be making. Please let me know if this assessment feels incorrect to you.
You can get wronger faster by using complex generators than compact generators.
… except that you have a natural immunity (well, aversion) to adopting complex generators, and a natural affinity for simple explanations. Or at least I think both of those are true of most people.
This comment feels to me like it might be dancing around saying “Hey! Don’t rape people! Make sure you are not raping people! You are saying some pretty rapey things”
Nope, that’s all coming from your expectations, not from me.
If I’d wanted to say that, I’d have said it. In fact, somebody had already said that. I actually downvoted it because I didn’t think the inference was particularly justified by the original text.
… except that you have a natural immunity (well, aversion) to adopting complex generators, and a natural affinity for simple explanations. Or at least I think both of those are true of most people.
It seems pretty important to me to distinguish between “heuristic X is worse than its inverse” and “heuristic X is better than its inverse, but less good than you think it is”.
Your top-level comment seemed to me like it was saying that a given simple explanation is less likely to be true than a given complex explanation. Here, you seem to me like you’re saying that simple explanation is more likely to be true, but people have a preference for them that is stronger than the actual effect, and so you want to push people back to having a preference that is weaker but still in the original direction.
a natural immunity (well, aversion) to adopting complex generators, and a natural affinity for simple explanations
I think this is wrong in an important way… most people find math complex, even when it describes simple things, and they find (simple) human and animal behaviours and emotions simple, even though they are some of the most complex natural phenomena I am aware of. So a more accurate statement would be “people have biases in the space of possible explanations that sometimes lead them toward overly complex explanations and sometimes lead them to overly simple explanations”.
Then we could recover your original claim that in the space of explanations of human behaviour, people are more likely to look for overly simple explanations than for overly complex explanations. But I think in explaining human behaviour people often do both, and when looking for sufficiently complex explanations, it is still true that more of them are wrong than are right.
Nope, that’s all coming from your expectations, not from me.
Right. Sorry if it caused any offence. People often seem motivated to misunderstand and condemn others, so I wouldn’t fault anyone for not wanting to say what they truly mean when discussing topics like these.
I think there may be more truth in what you’re saying than a lot of people would be entirely comfortable admitting… although what’s a cause and what’s an effect is hard to say.
… but I also think that going around looking for “compact generators” of human behavior, especially social behavior, is basically asking to be wrong. In fact, you can apply a sort of anti-parsimony: if a theory claims to compactly generate any significant set of social dynamics, that’s evidence against the theory. People are constantly coming up with simple explanations, and they’re constantly turning out to be wrong, and any given simple explanation has to overcome that prior.
A whole lot of things are allowed to be going on at the same time, pushing toward the same or similar results, or more likely sometimes pushing in the same direction and at other times conflicting. There are allowed to be arbitrarily complicated networks of both positive and negative feedback. Which things are most important is allowed to change not just from person to person, but from time to time.
So, there’s this thing called Solomonoff induction. It works, provably, for anything Turing computable. And human social behavior is definitely Turing computable.
“If a theory claims to compactly generate any significant set of social dynamics, that’s evidence against the theory” is an anti-inductive prior. It’s like saying that things which have happened less often before are more likely in the future, and therefore the sun will certainly not rise tomorrow.
A thing in the neighbourhood which sorta looks like an anti-parsimony prior is an “anti-wishful-thinking prior”. That is, we might think that humans sometimes have a tendency to latch onto overly simply theories and discard some of the data they should be explaining, so when someone proposes a theory that seems overly simple relative to the expected complexity of the domain they’re trying to explain, it could be rational to update that they’re likely engaging in “wishful thinking”, when compared to someone with a more complex theory. e.g. if someone posts a proof of the Riemann Hypothesis that’s ten pages long, or a very simple analysis of a trend in the stock market purported to give huge gains, etc....
It’s not anti-inductive to believe that sets of phenomena in [some domain] are unlikely to have simple explanations at [some level of abstraction].
WRT social behavior, it does seem to me that ‘a whole lot of things pushing toward the same or similar results’ is remarkably common, and the OP is too dismissive of other explanations, once it’s thought of one, for my priors. (At the same time, I totally think [simple theories that help explain significant sets of dynamics] are real.)
As alluded to by the name of the website, part of Solomonoff/MDL is that there doesn’t necessarily have to be a unique “correct” explanation: theories are better to the extent that their predictions pay for their complexity. It’s not that compact generators are necessarily “true”; it’s that if a compact generator is yielding OK predictions, then more complex theories need to be making better predictions to single themselves out. You shouldn’t say that looking for compact generators of a complex phenomenon is asking to be wrong unless you have a way to be less wrong.
IME in this domain, in epistemology-for-humans terms that may or may not translate easily into Solomonoff/MDL, taking a compact generator of a complex phenomenon too seriously — like, concentrating probability too strongly on its predictions, not taking anomalies seriously enough or looking hard enough for them, insufficiently expecting there to be more to say that sounds different in kind — is asking to be wrong, and not doing that is a way to be less wrong.
(Looking for compact generators but not taking them too seriously is good, but empirically seems to require more skill or experience.)
You don’t need to single out a specific complex theory to say “this simple theory is concentrating probability too strongly”, or to expect there to be some complex theory that pays for itself.
Solomonoff induction gives you a weighted sum over an infinite number of programs [1] . That’s not compact. And if were computable, which it isn’t, or even approximable, which it probably isn’t for this case, I doubt you’d be able to collect enough data in your lifetime for it to converge to speak of. Not even assuming that you were able to reliably collect all relevant data, which you’re not, and that you were actually encoding or processing the data in a formal way, which you’re also not.
And if you actually did somehow get your hands around a Solomonoff sum, you still wouldn’t be able to just grab a single term out of it, not even the one for the shortest program, and substitute it as “the” explanation on the grounds that “Solomonoff induction works”.
I can understand “compact generation” as a metaphorical allusion to Occam, but seriously, Solomoff induction isn’t even useful as a metaphor for any well-chosen approach here. You can’t let formalisms like that invade your thinking to the point where you seriously think in terms of them in areas where it doesn’t make sense.
Also, human social behavior probably isn’t deterministically Turing computable even if you model the entire universe. Probabilistically computable, probably, yes. In theory. And to be fair I’m sure Solomonoff goes through just fine to nondeterministic Turing processes. But anyway, you don’t actually have, and can’t actually get, a machine that computes human behavior or even a meaningful approximation to it.
There’s also no anti-inductive prior involved. What I’m saying isn’t about the underlying phenomena at all, and certainly doesn’t say that there’s no regularity in them. It’s about the theory, and it has in fact happened, far more often than not in my experience, that simple, single-explanation, “compact” theories, yield really bogus results.
Which is actually capable of encoding “lots of different, interacting things are going on” in a way that a single, deterministic Turing program would not be.
The point is that the sum is inversely weighted by compactness, not that the sum is itself compact.
i notice that it’s long been dogma that compact generators for human CEV are instantly and totally distrustworthy in exactly this way, that their compactness is very strong evidence against them
this feels related, but i’m not actually willing to stick my neck out and say it’s 1:1
I’d say that different people want/do different things (otherwise we probably wouldn’t have this debate right now; no one makes LW posts with convincing evolutionary arguments for why people breathe), therefore you should be suspicious of theories that predict that everyone wants the same thing.
You can get wronger faster by using complex generators than compact generators.
Matching the complexity of the hypothesis to expected complexity given the domain seems like an interesting idea.
“People and situations being different from one another so you should understand them carefully” seem like an important and true message, worth belabouring, even though it seems clear to me johnswentworth already knows and acknowledges this.
This comment feels to me like it might be dancing around saying “Hey! Don’t rape people! Make sure you are not raping people! You are saying some pretty rapey things!” which feels like a more useful message, but one that feels more unpleasant to be making. Please let me know if this assessment feels incorrect to you.
… except that you have a natural immunity (well, aversion) to adopting complex generators, and a natural affinity for simple explanations. Or at least I think both of those are true of most people.
Nope, that’s all coming from your expectations, not from me.
If I’d wanted to say that, I’d have said it. In fact, somebody had already said that. I actually downvoted it because I didn’t think the inference was particularly justified by the original text.
It seems pretty important to me to distinguish between “heuristic X is worse than its inverse” and “heuristic X is better than its inverse, but less good than you think it is”.
Your top-level comment seemed to me like it was saying that a given simple explanation is less likely to be true than a given complex explanation. Here, you seem to me like you’re saying that simple explanation is more likely to be true, but people have a preference for them that is stronger than the actual effect, and so you want to push people back to having a preference that is weaker but still in the original direction.
I think this is wrong in an important way… most people find math complex, even when it describes simple things, and they find (simple) human and animal behaviours and emotions simple, even though they are some of the most complex natural phenomena I am aware of. So a more accurate statement would be “people have biases in the space of possible explanations that sometimes lead them toward overly complex explanations and sometimes lead them to overly simple explanations”.
Then we could recover your original claim that in the space of explanations of human behaviour, people are more likely to look for overly simple explanations than for overly complex explanations. But I think in explaining human behaviour people often do both, and when looking for sufficiently complex explanations, it is still true that more of them are wrong than are right.
Right. Sorry if it caused any offence. People often seem motivated to misunderstand and condemn others, so I wouldn’t fault anyone for not wanting to say what they truly mean when discussing topics like these.