Thanks for the portraits; I appreciate getting to read it. I’m curious what would happen if you got one of them to read “The Elephant in the Brain”. (No idea if it’d be good or bad. Just seems like it might have some chance at causing something different.)
AnnaSalamon
Karma: 27,595
I continue to think CFAR is among the best places to donate re: turning money into existential risk reduction (including this year—basically because our good done seems almost linear in the number of free-to-participants programs we can run (because those can target high-impact AI stuff), and because the number of free-to-participants programs we can run is more or less linear in donations within the range in which donations might plausibly take us). If anyone wants my take on how this works, or on our last year or our upcoming year or anything like that, I’d be glad to talk: anna at rationality dot org.
RyanCarey writes:
If you are someone of median intelligence who just want to carry out a usual trade like making shoes or something, you can largely get by with recieved wisdom.
AFAICT, this only holds if you’re in a stable sociopolitical/economic context—and, more specifically still, the kind of stable sociopolitical environment that provides relatively benign information-sources. Examples of folks who didn’t fall into this category: (a) folks living in eastern Europe in the late 1930′s (especially if Jewish, but even if not; regardless of how traditional their trade was); (b) folks living in the Soviet Union (required navigating a non-explicit layer of recieved-from-underground knowledge); (c) folks literally making shoes during time-periods in which shoe-making was disrupted by the industrial revolution. It is to my mind an open question whether any significant portion of the US/Europe/etc. will fall into the “can get by largely with received wisdom” reference class across the next 10 years. (They might. I just actually can’t tell.)
“Flinching away from truth” is often about *protecting* the epistemology
Further discussion of CFAR’s focus on AI safety, and the good things folks wanted from “cause neutrality”
This is fair; I had in mind basic high school / Newtonian physics of everyday objects. (E.g., “If I drop this penny off this building, how long will it take to hit the ground?”, or, more messily, “If I drive twice as fast, what impact would that have on the kinetic energy with which I would crash into a tree / what impact would that have on how badly deformed my car and I would be if I crash into a tree?”).
We would indeed love to help those people train.
Yes. Or will seriously attempt this, at least. It seems required for cooperation and good epistemic hygiene.
Thanks; good point; will add links.
In case there are folks following Discussion but not Main: this mission statement was released along with:
Oh, sorry, the two new docs are posted and were in the new ETA:
http://lesswrong.com/lw/o9h/further_discussion_of_cfars_focus_on_ai_safety/ and http://lesswrong.com/r/discussion/lw/o9j/cfars_new_mission_statement_on_our_website/
CFAR’s new mission statement (on our website)
Apologies; the link is broken and I’m not sure how to edit or delete it; real link is: http://rationality.org/about/mission
Thanks for the thoughts; I appreciate it.
I agree with you that framing is important; I just deleted the old ETA. (For anyone interested, it used to read:
ETA: Having talked just now to people at our open house, I would like to clarify: Even though our aim is explicitly AI Safety...
CFAR does still need an art of rationality, and a community of rationality geeks that support that. We will still be investing at least some in that community. We will also still be running some “explore” workshops of different sorts aiming at patching gaps in the art (funding permitting), not all of which will be deliberately and explicitly backchained form AI Safety (although some will). Play is generative of a full rationality art. (In addition to sometimes targeting things more narrowly at particular high-impact groups, and otherwise more directly backchaining.) (More in subsequent posts.)I’m curious where our two new docs leave you; I think they make clearer that we will still be doing some rationality qua rationality.
Will comment later re: separate organizations; I agree this is an interesting idea; my guess is that there isn’t enough money and staff firepower to run a good standalone rationality organization in CFAR’s stead, and also that CFAR retains quite an interest in a standalone rationality community and should therefore support it… but I’m definitely interested in thoughts on this.
Julia will be launching a small spinoff organization called Convergence, facilitating double crux conversations between EAs and EA-adjacent people in, e.g., tech and academia. It’ll be under the auspices of CFAR for now but will not have opinions on AI. I’m not sure if that hits any of what you’re after.
I think we’ll have it all posted by Dec 18 or so, if you want to wait and see. My personal impression is that MIRI and CFAR are both very good buys this year and that best would be for each to receive some donation (collectively, not from each person); I expect the care for MIRI to be somewhat more straight-forward, though.
I’d be happy to Skype/email with you or anyone re: the likely effects of donating to CFAR, especially after we get our posts up.
No, it was just a pun. I believe trying to improve predictive accuracy is better than trying to promote view X (for basically any value of “X”); which I was hoping the pun of “Brier Boosting” off “Signal Boosting” would point to; but not Briers Score as such.
Coming up. Working on a blog post about it; will probably have it up in ~4 days.
Thanks!
I used to make the argument in the OP a lot. I applied it (among other applications) to Michael Vassar, who many people complained to me about (“I can’t believe he made obviously-fallacious argument X; why does anybody listen to him”), and who I encouraged them to continue listening to anyhow. I now regret this.
Here are the two main points I think past-me was missing:
1. Vetting and common knowledge creation are important functions, and ridicule of obviously-fallacious reasoning plays an important role in discerning which thinkers can (or can’t) help fill these functions.
(Communities — like the community of physicists, or the community of folks attempting to contribute to AI safety — tend to take a bunch of conclusions for granted without each-time-reexamining them, while trying to add to the frontier of knowledge/reasoning/planning. This can be useful, and it requires a community vetting function. This vetting function is commonly built via having a kind of “good standing” that thinkers/writers can be ruled out of (and into), and taking a claim as “established knowledge that can be built on” when ~all “thinkers in good standing” agree on that claim.
I realize the OP kind-of acknowledges this when discussing “social engineering”, so maybe the OP gets this right? But I undervalued this function in the past, and the term “social engineering” seems to me dismissive of a function that in my current view contributes substantially to a group’s ability to produce new knowledge.)
2. Even when a reader is seeking help brainstorming hypotheses (rather than vetting conclusions), they can still be lied-to and manipulated, and such lies/manipulations can sometimes disrupt their thinking for long and costly periods of time (e.g., handing Ayn Rand to the wrong 14-year-old; or, in my opinion, handing Michael Vassar to a substantial minority of smart aspiring rationalists). Distinguishing which thinkers are likely to lie or manipulate is a function more easily fulfilled by a group sharing info that rules thinkers out for past instances of manipulative or dishonest tactics (rather than by the individual listener planning to ignore past bad arguments and to just successfully detect every single manipulative tactic on their own).
So, for example, Julia Galef helpfully notes a case where Steven Pinker straightforwardly misrepresents basic facts about who said what. This is helpful to me in ruling out Steven Pinker as someone who I can trust not to lie to me about even straightforwardly checkable facts.
Similarly, back in 2011, a friend complained to me that Michael would cause EAs to choose the wrong career paths by telling them exaggerated things about their own specialness. This matched my own observations of what he was doing. Michael himself told me that he sometimes lied to people (not his words) and told them that the thing that would most help AI risk from them anyhow was for them to continue on their present career (he said this was useful because that way they wouldn’t rationalize that AI risk must be false). Despite these and similar instances, I continued to recommend people talk to him because I had “ruled him in” as a source of some good novel ideas, and I did this without warning people about the rest of it. I think this was a mistake. (I also think that my recommending Michael led to considerable damage over time, but trying to establish that claim would require more discussion than seems to fit here.)
To be clear, I still think hypothesis-generating thinkers are valuable even when unreliable, and I still think that honest and non-manipulative thinkers should not be “ruled out” as hypothesis-sources for having some mistaken hypotheses (and should be “ruled in” for having even one correct-important-and-novel hypothesis). I just care more about the caveats here than I used to.