Generally, there is a substantial literature on the topic within the field of network science. The right keywords for Google scholar are something like spreading dynamics in complex networks. Information cascades does not seem to be the best choice of keywords.
There are many options how you can model the state of the node (discrete states, oscillators, continuous variables, vectors of anything of the above,...), multiple options how you may represent the dynamics (something like Ising model / softmax, versions of voter model, oscillator coupling, …) and multiple options how you model the topology (graphs with weighted or unweighted edges, adaptive wiring or not, topologies based on SBM, or scale-free networks, or Erdős–Rényi, or Watts-Strogatz, or real-world network data,… This creates somewhat large space of options, which were usually already explored somewhere in the literature.
What is possibly the single most important thing to know about this, there are universality classes of systems which exhibit similar behaviour; so you can often ignore the details of the dynamics/topology/state representation.
Overall I would suggest to approach this with some intellectual humility and study existing research more, rather then try to reinvent large part of network science on LessWrong. (My guess is something like >2000 research years were spent on the topic often by quite good people.)
I haven’t looked through your links in much detail, but wanted to reply to this:
Overall I would suggest to approach this with some intellectual humility and study existing research more, rather then try to reinvent large part of network science on LessWrong. (My guess is something like >2000 research years were spent on the topic often by quite good people.)
I either disagree or am confused. It seems good to use resources to outsource your ability to do literature reviews, distillation or extrapolation, to someone with higher comparative advantage. If the LW question feature can enable that, it will make the market for intellectual progress more efficient; and I wanted to test whether this was so.
I am not trying to reinvent network science, and I’m not that interested in the large amount of theoretical work that has been done. I am trying to 1) apply these insights to very particular problems I face (relating to forecasting and more); and 2) think about this from a cost-effectiveness perspective.
I am very happy to trade money for my time in answering these questions.
(Neither 1) nor 2) seems like something I expect the existing literature to have been very interested in. I believe this for similar reasons to those Holden Karnofsky express here.)
I was a bit confused by we but aren’t sure how to reason quantitatively about the impacts, and how much the LW community could together build on top of our preliminary search, which seemed to nudge toward original research. Outsourcing literature reviews, distillation or extrapolation seem great.
Agreed. I realise the OP could be misread; I’ve updated the first paragraph with an extra sentence mentioning that summarising and translating existing work/literature in related domains is also really helpful.
Thanks for the pointers to network science Jan, I don’t know this literature, and if it’s useful here then I’m glad you understand it well enough to guide us (and others) to key parts of it. I don’t see yet how to apply it to thinking quantitatively about scientific and forecasting communities.
If you (or another LWer) thinks that the theory around universality classes is applicable in thinking about how to ensure good info propagation in e.g. a scientific community, and you’re right, then I (and Jacob and likely many others) would love to read a summary, posted here as an answer. Might you explain how understanding the linked paper on universality classes has helped you think about info propagation in forecasting communities / related communities? Concrete heuristics would be especially interesting
(Note that Jacob and I have not taken a math course in topology or graph theory and won’t be able to read answers that assume such, though we’ve both studied formal fields of study and could likely pick it up quickly if it seemed practically useful.)
In general we’re not looking for *novel* contributions. To give an extreme example, if one person translates an existing theoretical literature into a fully fleshed out theory of info-cascades for scientific and forecasting communities, we’ll give them the entire prize pot.
Short summary of how is the lined paper important: you can think about bias as some sort of perturbation. You are then interested in the “cascade of spreading” of the perturbation, and especially factors like the distribution of sizes of cascades. The universality classes tell you this can be predicted by just a few parameters (Table 1 in the linked paper) depending mainly on local dynamic (forecaster-forecaster interactions). Now if you have a good model of the local dynamic, you can determine the parameters and determine into which universality class the problem belongs. Also you can try to infer the dynamics if you have good data on your interactions.
I’m afraid I don’t know enough about how “forecasting communities” work to be able to give you some good guesses what may be the points of leverage. One quick idea, if you have everybody on the same platform, may be to do some sort fo A/B experiment—manipulate the data so some forecasters would see the predictions of other with an artificially introduced perturbation, and see how their output will be different from the control group. If you have data on “individual dynamics” liken that, and some knowledge of network structure, the theory can help you predict the cascade size distribution.
(I also apologize for not being more helpful, but I really don’t have time to work on this for you.)
Information cascades does not seem to be the best choice of keywords.
I wouldn’t say that ‘information cascades’ isn’t the best choice of keywords. What’s happening here is that the same phenomenon is studied by different disciplines in relative isolation from each other. As a consequence, the phenomenon is discussed under different names, depending on the discipline studying it. ‘Information cascades’ (or, as it is sometimes spelled, ‘informational cascades’) is the name used in economics, while network science seems to use a variety of related expressions, such as the one you mention.
It seems to offer a learnt summary of the relevance of network science (which offers a complementary perspective on the phenomenon to the microeconomic literature linked by other commenters), which not implausibly took Jan at least an order of magnitude less time to compile than it would have taken us. (For example, the seemingly simple fact of using a different Google scholar keyword than “information cascade” might have taken several hours to realise for a non-expert.)
It also attempts to apply these to the case of forecasting (despite Jan’s limited knowledge of the domain), which is a task that would likely have been even harder to do without deep experience of the field.
Generally, there is a substantial literature on the topic within the field of network science. The right keywords for Google scholar are something like spreading dynamics in complex networks. Information cascades does not seem to be the best choice of keywords.
There are many options how you can model the state of the node (discrete states, oscillators, continuous variables, vectors of anything of the above,...), multiple options how you may represent the dynamics (something like Ising model / softmax, versions of voter model, oscillator coupling, …) and multiple options how you model the topology (graphs with weighted or unweighted edges, adaptive wiring or not, topologies based on SBM, or scale-free networks, or Erdős–Rényi, or Watts-Strogatz, or real-world network data,… This creates somewhat large space of options, which were usually already explored somewhere in the literature.
What is possibly the single most important thing to know about this, there are universality classes of systems which exhibit similar behaviour; so you can often ignore the details of the dynamics/topology/state representation.
Overall I would suggest to approach this with some intellectual humility and study existing research more, rather then try to reinvent large part of network science on LessWrong. (My guess is something like >2000 research years were spent on the topic often by quite good people.)
I haven’t looked through your links in much detail, but wanted to reply to this:
I either disagree or am confused. It seems good to use resources to outsource your ability to do literature reviews, distillation or extrapolation, to someone with higher comparative advantage. If the LW question feature can enable that, it will make the market for intellectual progress more efficient; and I wanted to test whether this was so.
I am not trying to reinvent network science, and I’m not that interested in the large amount of theoretical work that has been done. I am trying to 1) apply these insights to very particular problems I face (relating to forecasting and more); and 2) think about this from a cost-effectiveness perspective.
I am very happy to trade money for my time in answering these questions.
(Neither 1) nor 2) seems like something I expect the existing literature to have been very interested in. I believe this for similar reasons to those Holden Karnofsky express here.)
I was a bit confused by we but aren’t sure how to reason quantitatively about the impacts, and how much the LW community could together build on top of our preliminary search, which seemed to nudge toward original research. Outsourcing literature reviews, distillation or extrapolation seem great.
Agreed. I realise the OP could be misread; I’ve updated the first paragraph with an extra sentence mentioning that summarising and translating existing work/literature in related domains is also really helpful.
Thanks for the pointers to network science Jan, I don’t know this literature, and if it’s useful here then I’m glad you understand it well enough to guide us (and others) to key parts of it. I don’t see yet how to apply it to thinking quantitatively about scientific and forecasting communities.
If you (or another LWer) thinks that the theory around universality classes is applicable in thinking about how to ensure good info propagation in e.g. a scientific community, and you’re right, then I (and Jacob and likely many others) would love to read a summary, posted here as an answer. Might you explain how understanding the linked paper on universality classes has helped you think about info propagation in forecasting communities / related communities? Concrete heuristics would be especially interesting
(Note that Jacob and I have not taken a math course in topology or graph theory and won’t be able to read answers that assume such, though we’ve both studied formal fields of study and could likely pick it up quickly if it seemed practically useful.)
In general we’re not looking for *novel* contributions. To give an extreme example, if one person translates an existing theoretical literature into a fully fleshed out theory of info-cascades for scientific and forecasting communities, we’ll give them the entire prize pot.
Short summary of how is the lined paper important: you can think about bias as some sort of perturbation. You are then interested in the “cascade of spreading” of the perturbation, and especially factors like the distribution of sizes of cascades. The universality classes tell you this can be predicted by just a few parameters (Table 1 in the linked paper) depending mainly on local dynamic (forecaster-forecaster interactions). Now if you have a good model of the local dynamic, you can determine the parameters and determine into which universality class the problem belongs. Also you can try to infer the dynamics if you have good data on your interactions.
I’m afraid I don’t know enough about how “forecasting communities” work to be able to give you some good guesses what may be the points of leverage. One quick idea, if you have everybody on the same platform, may be to do some sort fo A/B experiment—manipulate the data so some forecasters would see the predictions of other with an artificially introduced perturbation, and see how their output will be different from the control group. If you have data on “individual dynamics” liken that, and some knowledge of network structure, the theory can help you predict the cascade size distribution.
(I also apologize for not being more helpful, but I really don’t have time to work on this for you.)
I wouldn’t say that ‘information cascades’ isn’t the best choice of keywords. What’s happening here is that the same phenomenon is studied by different disciplines in relative isolation from each other. As a consequence, the phenomenon is discussed under different names, depending on the discipline studying it. ‘Information cascades’ (or, as it is sometimes spelled, ‘informational cascades’) is the name used in economics, while network science seems to use a variety of related expressions, such as the one you mention.
We (jacobjacob and Ben Pace) decided to award $200 (out of the total bounty of $800) to this answer (and the additional comment below).
It seems to offer a learnt summary of the relevance of network science (which offers a complementary perspective on the phenomenon to the microeconomic literature linked by other commenters), which not implausibly took Jan at least an order of magnitude less time to compile than it would have taken us. (For example, the seemingly simple fact of using a different Google scholar keyword than “information cascade” might have taken several hours to realise for a non-expert.)
It also attempts to apply these to the case of forecasting (despite Jan’s limited knowledge of the domain), which is a task that would likely have been even harder to do without deep experience of the field.
I’ll PM Jan about payment details.