Information-generating research projects
The research projects that I’m interested in here are projects that:
Involve aggregating and analyzing information
Don’t require experiments (e.g. not biomedical research)
Can be done by people who aren’t subject matter experts.
Lead to publicly accessible information (not proprietary research in finance)
Some examples:
GiveWell has generated information relevant to finding outstanding giving opportunities. The value of this information extends beyond giving opportunities: for example, GiveWell’s investigation of biomedical research as a philanthropic cause is relevant to those considering a career in the field.
80,000 Hours has been doing interviews, collecting salary information, and doing cost-effectiveness analyses to help inform people’s career choices.
MIRI has been generating information on the historical record of predictions of the medium term future, the historical record of efforts to influence the medium term future, the state of the artificial intelligence field, and more.
What are some other examples of topics for which research of this type could have high impact?
Note: I formerly worked as a research analyst at GiveWell and have done contract work for 80000 Hours and MIRI.
I think that one incredibly important task is to give impartial and reliable information about the quality of information. Not even scientific peer-review is a sufficiently good procedure. For this reason, the Meta-Research Innovation Centre at Stanford has started a:
To do these kinds of evaluations you of course need expertise. But other kinds of evaluations require little but intelligence, good judgment, an impartial mindset and persistence. For instance, we crave for better and more impartial evaluations of the validity of statements made by governments, politicians, companies, and other influential organizations. I have a long-term goal of doing something along those lines. I think rationalists could contribute a lot here.
Thanks! Can you give some examples of what you have in mind? There are already some efforts like FactCheck.org. Are there specific areas where you see room for improvement?
These too: [factmata.com](https://factmata.com/), [fullfact.org](https://fullfact.org/) (I know the latter a bit and like it a lot). These are for media sector though not academia.
Thanks I didn’t know of Factcheck though I know newspapers sometimes do similar things.
I think that partly due to the existence of factcheckers, and partly because it’s so easy to check for yourself whether a certain statement of fact is true, it is far less common that politicians make false statements of fact than that they fool the voters in other ways. For instance, a typical political debate consists of one logical fallacy after the other—strawmen, guilt by association, invalid ad hominems etc. I think that it would not be too hard to provide the voters with a systematic and useful analysis of these fallacies. Hopefully that would shame the politicians into better behaviour.
There are many other arguments one could criticize, though: banks arguments for why people should buy expensive products rather than cheap index funds, bad arguments in the news media, etc. What I think would be important would be too remain cool-headed and impartial. If you aren’t, you’ll just be another political player. What the public debate needs is impartial “referees” who criticize invalid reasoning. Ideally, these referees would be smart and altruistic citizens—much like those working at Wikipedia. One idea would be the construction of a wiki for criticism—Criticopedia or Wikicriticism (perhaps with a Karma system where e.g. presidential speeches would be rated on the basis of argumentative soundness). The ultimate goal would be the End of Bullshit.
Australia has an Australian version of factcheck too. However, it only goes after things which are the communities current expectations, not future expectations. Therefore, it understates risks to future ethical injunctions. For instance, politicians are under little or no official nor social pressure to disclose their stock holdings despite legislating on matters that can be expected to dramatically affect their prices in predictable ways. I, for one, intend to sell my shares before I gain any kind of position where someone would have reason to believe I’d have a conflict on interest. For the sake of easy management, I’ll probably sell all my securities and just let it all sit in 1 long term and 1 immediate use account (+ compulsory accounts like superannuation) in my countries main bank (Commonwealth bank) just to make my life easier and raise credibility.
Thanks for the further thoughts.
I have my suspicions about the second point: “Can be done by people who aren’t subject matter experts.” The whole concept of doing research projects in areas about which one knows little doesn’t strike me as a good idea. GIGO is a real problem not solvable by fancy number manipulations.
I do like cross-disciplinary raids and excursions, but they involve bringing the tools, techniques, and insights of one discipline to another field and not just amateurs breezing in.
I have in mind the people organizing what subject matter experts have written.
I think the idea of trying to get people who aren’t experts to aggregate and analyze information is an excellent one. I think one good thing to do is to think about what kinds of analyses/aggregations that aren’t done at present you could do. What does science/Wikipedia/Google etc not cover?
I think one major problem is that when there is a conflict of interest, no one provides us with impartial, reliable information. That is why I suggest that there should be a place for reliable and impartial criticism of statements. But one could also think of more positive suggestions; for instance for how to take advantage of information technology in a socially optimal fashion. I think that at the moment, special interest groups are blocking us from making optimal use of the new technology. For instance, in my field, the academia, there is a massive resistance to massive online courses because academics fear it will lead to unemployment (which I don’t think is true, by the way, but that is beside the point), even though these massive online courses could (or that’s what people hope at least) provide students with better education at a lower price.
My point is that in cases where a certain policy P maximizes the total utility, but a certain alternative policy Q leads to much better outcomes to a special interest group, and just a little bit worse outcome to everyone else, Q will often chosen (even though there are no good reasons for it) because the special interest group will care so much more about the issue than everyone else. Hence it’s important that someone takes responsibility to work for the common good.
To a certain extent, I think there are think tanks that does the latter, though, but it would be good if one could involve more people in this (for instance on fora or wikis).