Jonah,
Thanks for expressing an interest in donating to SIAI.
(a) SIAI has secured a 2 star donation from GiveWell for donors who are interested in existential risk.
I assure you that we are very interested in getting the GiveWell stamp of approval. Michael Vassar and Anna Salamon have corresponded with Holden Karnofsky on the matter and we’re trying figure out the best way to proceed.
If it were just a matter of SIAI becoming more transparent and producing a larger number of clear outputs I would say that it is only a matter of time. As it stands, GiveWell does not know how to objectively evaluate activities focused on existential risk reduction. For that matter, neither do we, at least not directly. We don’t know of any way to tell what percentage of worlds that branch off from this one go on to flourish and how many go on to die. If GiveWell decides not to endorse charities focused on existential risk reduction as a general policy, there is little we can do about it. Would you consider an alternative set of criteria if this turns out to be the case?
We think that UFAI is the largest known existential risk and that the most complete solution—FAI—addresses all other known risks (as well as the goals of every other charitable cause) as a special case. I don’t mean to imply that AI is the only risk worth addressing at the moment, but it certainly seems to us to be the best value on the margin. We are working to make the construction of UFAI less likely through outreach (conferences like the Summit, academic publications, blog posts like The Sequences, popular books and personal communication) and make the construction of FAI more likely through direct work on FAI theory and the identification and recruitment of more people capable of working on FAI. We’ve met and worked with several promising candidates in the past few months. We’ll be informing interested folk about our specific accomplishments in our new monthly newsletter, the June/July issue of which was sent out a few weeks ago. You can sign up here.
(b) You publically apologize for and qualify your statements quoted by XiXiDu here. I believe that these statements are very bad for public relations. Even if true, they are only true at the margin and so at very least need to be qualified in that way.
It would have been a good idea for you to watch the videos yourself before assuming that XiXiDu’s summaries (not actual quotes, despite the quotation marks that surrounded them) were accurate. Eliezer makes it very clear, over and over, that he is speaking about the value of contributions at the margin. As others have already pointed out, it should not be surprising that we think the best way to “help save the human race” is to contribute to FAI being built before UFAI. If we thought there was another higher-value project then we would be working on that. Really, we would. Everyone at SIAI is an aspiring rationalist first and singularitarian second.
First of all, thank you so much for posting this. I’ve been contemplating composing a similar post for a while now but haven’t because I did not feel like my experience was sufficiently extensive or my understanding was sufficiently deep. I eagerly anticipate future posts.
That said, I’m a bit puzzled by your framing of this domain as “arational.” Rationality, at least as LW has been using the word, refers to the art of obtaining true beliefs and making good decisions, not following any particular method. Your attitude and behavior with regard to your “mystical” experiences seems far more rational than both the hasty enthusiasm and reflexive dismissal that are more common. Most of what my brain does might as well be magic to me. The suggestion that ideas spoken to you by glowing spirit animals should be evaluated in much the same way as ideas that arise in less fantastic (though often no less mysterious) ways seems quite plausible and worthy of investigation. You seem to have done a good job at keeping your eye on the ball by focusing on the usefulness of these experiences without accepting poorly thought out explanations of their origins.
It may be the case that we have the normative, mathematical description of what rationality looks like down really well, but that doesn’t mean we have a good handle on how best to approximate this using a human brain. My guess is that we’ve only scratched the surface. Peak or “mystical” experiences, much like AI and meta-ethics, seem to be a domain in which human reasoning fails more reliably than average. Applying the techniques of X-Rationality to this domain with the assumption that all of reality can be understood and integrated into a coherent model seems like a fun and potentially lucrative endeavor.
So now, in traditional LW style, I shall begin my own contribution with a quibble and then share some related thoughts:
As far as I understand it, the Buddhist claim is that the unenlightened mind fails to understand the nature of one particular aspect of reality: it’s own experience of the world and relationship to it. One important goal of what is typically called “insight meditation” seems to be to cause people to really grok that the map is not the territory when it comes to the category of “self.” What follows is my own, very tentative, model of “enlightenment”:
By striving to dissect your momentary experience in greater and greater detail, the process by which certain experiences are labeled “self” and others “not-self” becomes apparent. It also becomes apparent that the creation of this sense of a separate self is at least partially responsible for the rejection of or “flinching away” from certain aspects of your sensory experience and that this is one of the primary causes of suffering (which seems to be something like “mental conflict”). My understanding of “enlightenment” is as the final elimination (rather than just suppression) of this tendency to “shoot the messenger.” This possibility is extremely intriguing to me because it seems like it should eliminate not only suffering but what might be the single most important source of “wireheading” behaviors in humans. People who claim to have achieved it say it’s about as difficult as getting an M.D. Seems worthy of investigation.
Rather than go on and on here, I think it’s about time I organized my experience and research into a top-level post.