I’m currently researching forecasting and epistemics as part of the Quantified Uncertainty Research Institute.
ozziegooen(Ozzie Gooen)
Karma: 3,222
I would hope that if there were a public web application, it would take 20-100 different statistics, and allow people to choose which to privilege. Not sure if it’s the reader’s or the website’s responsibility to choose worthwhile stats to focus on, especially if they became standardized and other agencies were able to focus on what they wanted.
For example, I imagine that foundations could have different combination metrics, like “we found academics with over 15 papers with collectively 100 citations, which must average at least an 60% on a publicity scale and have a cost-to-influence index of over 45”. These criterion could be highly focussed on the needs and specialties of the foundation.
It seems very strange to me to look at e1 and e2 as point values without weights or confidence distributions. Taking no other information about e1 and e2 other than the fact that they both indicate a 999/1000 victory is quite limiting and makes for impossible meta-analysis. Other information could include what you think of both sources.
If you could get an accurate 90% confidence interval for each (or any accurate probability distribution) this could make a lot more sense. This must encompass the expected error in the New York Time’s error margin, especially if they don’t have one. For example, you may find that whenever reference a statistic, 90% of the time it is has at most 15% error compared to the true result (this would be really useful btw, someone should do this). Even if you estimate this number, you could still get a workable value.
If their 90% confidence interval was 0% error, and their reported statistics were always exactly true, then I do not believe you would update at all from e2.
I feel like it is possible to combine two 90% confidence intervals, and my guess is that any two with the same mean would result in higher certainty than at least the worst estimate (the one with the wider 90% confidence interval), possible higher than both. Solving this mathematically is something I’m not too sure about.
While I find this particular re-definition a bit silly, that doesn’t mean that in general having more succinct definitions isn’t a good thing.
If the second definition of evidence were used, it would mean that “collecting evidence” would be a fundamentally different thing than it would be in the first case. In the second, “evidence” is completely relative to what is already known, and lots of new material would not be included.
So if I go out and ask people to “collect evidence”, their actions should be different depending on which definition we collectively used.
In addition, the definition would lead to interesting differences in quantities. If we used the first definition, me having “lots of evidence” could mean having lots of redundant evidence for one small part of something (of course, it would also be helpful to quantify “lots”, but I believe that or something similar could be done). In the second definition, I imagine it would be much more useful to what I actually want.
This new definition makes “evidence” much more coupled to resulting probabilities, which in itself could be a good thing. However, it seems like an unintuitive stretch for my current understanding of the word, so I would prefer that rather than re-defining the word, a condition were used. For example, “updating evidence” for the second definition.
Sorry for posting this late, but I have done this as part of an existential risk meetup in San Francisco.
We experimented quite a bit, but here is one general algorithm for making fermi estimates as a group.
State the question (how many almonds are in that bag) and what people are allowed to see/measure (you can look at the bag, inspect it, can not touch).
Ask everyone to give their immediate intuitive 90% confidence interval. This in itself is essentially the smallest possible fermi estimate.
Give people around 2 minutes to sketch out something, and then give new estimates.
Spend another 8-15 minutes for a longer, more mathematical answer. Allow people to use calculators, spend some time writing out thoughts.
Go around the room, asking for estimates and the methods used to come up with them.
Try to collectively decide on one model. Have everyone give their inputs to that model. See if you can come up with an answer with higher certainty than any person could come up with individually.
Look at all of the results again, with the actual result (if possible). Compare & contrast.
General Recommendations
Use confidence intervals! As mentioned above, if you can do this, you will get much more interesting answers.
Try to graph out your models. I made an example here of a few ways to do this.
Compare answers and see if the group can come up with a consensus to first a best model, or second the actual number.
Consider what would help make a fermi question calculator program :) (I’m starting making one, having a difficult UI time)
Also, if together you guys hear of any good books or new methods for doing large Fermi estimates, please post.
I couldn’t believe the registration “questions”.
“A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?”
“If it takes 5 machines 5 minutes to make 5 widgets, how long would it take 100 machines to make 100 widgets?”
“In a lake, there is a patch of lily pads. Every day, the patch doubles in size. If it takes 48 days for the patch to cover the entire lake, how long would it take for the patch to cover half the lake?”
These are the exact questions from this blog post, to the letter (and are also all very well known outside that, but sometimes in different orders). What?
Then there were a few that state “Discontinued”, including one with the answers (this was multiple choice), “Completely a Fox”, “In between a Fox and a Hedgehog”, and “Completely a Hedgehog” (with a few others in between). Why these questions were not just removed from the form is beyond me. I’m also really interested in how, at least for a while, they may have assumed readers knew about this dichotomy, especially readers who they assumed did not know the 3 most popular analysis/rationality questions on the internet.
The main site seems to be written in Ember or Backbone.js or something, and yet it’s really tough to use and is very buggy in chrome. (Closer inspection shows they may have done it without a MVC js framework. Also, they haven’t minified their main JS files for some reason.)
The existing platform has all the markings of a big bureaucratic project combined with recent web and prediction market trends. Bizarre.
On that note, seriously, check out IARPA if you get a chance. It seems to be a recent (2008) offshoot of DARPA, that’s like “Darpa, but for Spies”. Their mission is to “invest in high-risk/high-payoff research programs that have the potential to provide the United States with an overwhelming intelligence advantage over future adversaries.” link
Basically, it seems like they are the government’s bleeding edge of spying and internet security. Some of their researchers seem to be doing projects focussed on quantum computing and computer security. Others on using public big data to predict possible threats. Others to train people to understand their cognitive biases by playing games. It’s almost as if someone cloned LessWrong and had them work for the government but closed sourced and significantly less idealistic.
It’s really a pity that it’s all closed, with the mission itself implying a zero-sum game. However, there is a surprising amount of public information on their website. For example, check out the complete sirius page. Not only can you find a pretty precise documentation of how the IARPA is trying to make this game to reduce cognitive biases, but you can also find all of the proposals by different agencies to get this contract. Really interesting stuff.
A low bar should still be helpful, but this is more of a google search than a rationality check. They could have at least changed the numbers slightly or the nouns by a bit. If it were to check if you were a computer, it wouldn’t work at all because the answers are the same each time.
Of course, answering them incorrectly is still a bad sign. I just imagine that with a 3 minutes of more effort (especially compared with the multi-year long complex web application) they could have had something substantially more useful and interesting.
“Monogamous research” makes a lot of sense for narrowly defined and relatively brute force research, but much less for vague fields. For example, within X-risk, we may need someone to write a popular book summary. For this it makes a lot of sense for a few experts to do this while the less knowledgable work on other things for the time being.
The fields you mention are of course much more vague though. My guess is that X-risk can be “polygamous-optimal”, but perhaps only so far that it can be broken up into “monogamous” pieces.
If a market is being highly influenced by people hedging their bets, then there is significant arbitrage opportunities for others to make money offsetting that. Prediction markets with decent liquidity should be fine, and in fact the presence of a group with hedging interests presents more opportunities for others to do this arbitrage, helping the market.
This makes me angry on so many levels. (Spoilers Below)
“We believe it is the goal of every species to behave with the best ethics possible.” → Who would say that? If only intelligent beings, is the argument that every single being a utilitarian, or only a small important subset? Do we argue that apes should be utilitarians? What do they mean the “goal”; isn’t the benefit of the ethics more the “goal” then the means are?
’The best ethics? It’s like… trying to maximize how happy everyone is.” Utilitarianism is here explained by someone who barely seems to understand utilitarianism. This is a straw man, that’s not fair. Any decent enthusiast should at least provide a much more succinct response, and I imagine that in the future we should have a much better understanding of it as well.
The alien plays sarcastic in the end, saying it’s “highest behavior” is “smugly tolerating inferior species”. I’m curious what Zach could have put their as a realistic possible example.
Short answer, it’s possible to make anything look silly when you get idiots to represent it.
Also, note that if we actually were at a point as a species where utilitarianism were the de-facto philosophy, my guess is that the world would look quite different. And I agree with Tuxedage on hedonistic utilitarianism vs. utilitarianism here.
Makerspaces? I hope you mean hackerspaces / coworking spaces. If you’re not familiar with them, I strongly recommend checking out http://hackerspaces.org/wiki/List_of_ALL_Hacker_Spaces . There are hackerspaces almost everywhere. I went to Singapore for study abroad, and because of the hackerspace got intro’d to a bunch of awesome people and other events.
Coworking spaces are an even more recent phenomenon. I get the feeling like they, on average, aren’t as social as hackerspaces, but they still provide some sort of community.
There’s also co-living places, one of which I’m in right now ( http://www.risesf.com/ ). In San Francisco there are a few of these, basically big collective houses that support 8-15 people. A “community in a box” so to speak.
I’ve also found that there are a bunch of “we’re the cool intellectual-ish people” groups out there. They seem to kind of blend together, but sometimes are international and very good for networking. The Sandbox network comes to mind (http://sandbox.is/). Also the TED and BIL communities, which are both pretty large.
Of course, the single biggest recommendation for any of this is that they are all significantly easier if you are located in or near a city. Seriously, that’s about the biggest “improvement hack” I’ve made so far in my life (coming from suburbia New York to San Francisco).
Meetup : San Francisco: Effective Altruism
Awesome map. I highly encourage the creation of diagrams and similar structures, I will reference this repeatedly. On that note, something like this (maybe iterated on a bit) would make a fantastic wall poster.
Do you have any calculations on how decision training is one of the highest-impact ways to improve the future? This seems very different from the other Effective Altruist organizations I’m aware of, and if there were a really convincing Fermi estimate or similar, it would mean a great deal to the community.
Creating a Text Shorthand for Uncertainty
I’m not suggesting that we make this a mandated LessWrong policy, but I think it may be fun to play with personally ~i4. I imagine that something more sensical would be created after some thought and work ~i2.
Also it could make a lot of sense for an organization to accept a standard like this rather than a blog community. For instance, many startups or EA orgs already use a lot of internal jargon ~e3i2, so something like this doesn’t seem like a clear negative in that light ~i2.
Concerning the appearance aspect, it may be possible to hide the phrases (similar to the markdown mention earlier) with a small symbol that one could “inspect” when interested.
Very good point. I like how this could go higher or have in-between values quite easily. In retrospect an equation like this makes much more sense than an intuitive guess, what I wrote down was mostly to use as a start.
I’m not sure if this is exactly the perfect equation for this, given that I think I’d probably want them to be a bit more spaced out if they went to 10 (going further in confidence past 99.9% perhaps) ~i1.
I feel like probabilities are longer (80 as opposed to 3) and in some ways contain more information (1 out of a 100) than we can really state ~i1. In effect, we have few significant figures.
That said, the understandability of percentages may outweigh this benefit (-~i1).
Not exactly. Counterfactual credit on individuals isn’t additive. For instance, we can say that almost every piece of the car is counterfactually (for the removal of the piece) essential (worth the entire value of the car).
That said, in this case, one would have to wonder what these employees would have done otherwise. I imagine the value of the founders for Google counterfactually, is somewhat equal to the value of Google minus the value of what the used-by-Google resources (money and lots of talent), had they been used elsewhere.
I definitely found out something similar. I’ve come to believe that most ‘popular science’, ‘popular history’ etc books are on audible, but almost anything with equations or code is not.
The ‘great courses’ have been quite fantastic for me for learning about the social sciences. I found out about those recently.
Occasionally I try podcasts for very niche topics (recent Rails updates, for instance), but have found them to be rather uninteresting in comparison to full books and courses.
About “how trusting should I be”, I would like to quickly bring up a bit of the scientific research. I’ve read a bit of this, and have concluded in high certainty that individual trust gives off a decent positive externality.
Quite a bit of econometric research (yea, it’s not the best, but it’s what we have) shows that positive levels of trust correlate with higher economic growth.
See: http://oep.oxfordjournals.org/content/56/1/118.short http://onlinelibrary.wiley.com/doi/10.1111/1468-0297.00609/abstract
However, I very much remember reading about how entrepreneurs have lower trust levels than most people on average, but can’t seem to find the paper right now.
What I’d imagine is that in general trust helps society, which makes a lot of sense as it allows trade to function. However, it may very well be that the level of trust optimal for an individual is quite lower than that optimal for his or her society. Makes sense if trust is looked at similar to altruism.
In this case, obviously the more trust people have with other people and their bags, the more cases of stealing that will occur. On the whole this may help your surrounding people, as some of these will benefit from stealing. However, the disadvantage would come to yourself at the expected cost of having your stuff stolen.
It would be interesting to factor in not only how much your bags being lost would cost you, but how much they would benefit the person who steals them.