You can donate it to my startup instead, our board of directors has just unanimously decided to adopt this name. Paypal is fine. Our mission is developing heuristics for personal income optimization.
Zetetic
Karma: 694
Well, here are my two cents. (1) It isn’t strictly correct to call him an AI researcher. A more correct classification would be something like AGI theorist; more accurate still would be FAI theorist. (2) Normal anomaly mentioned his TDT stuff, but of course that is only one of his papers. Will Newsome mentioned CFAI. I would add to that list the Knowability of FAI paper, his paper coauthored with Nick Bostrom, Coherent Extrapolated Volition, Artificial Intelligence as a Positive and Negative Factor in Global Risk, and LOGAI.
He (as I understand it, though, perhaps I am wrong about this) essentially invented the field (Friendly Artificial General Intelligence) as an area for substantial study and set out some basic research programs. The main one of which seems to be a decision theory for an agent in a general environment that is capable of overcoming the issues that current decision theories have; mainly that they do not always give the action that we would recognize as having the greatest utility relative to our utility function.
My biggest concern is that simply spreading the memes will be counterproductive if they are not spread properly (partial understanding would be a huge concern). If everyone is just guessing the teacher’s password, we will have probably done more damage than good.
I don’t know, this post feels a bit… ‘woo’ to me. Are there any peer-reviewed studies of the effects of meditation that you think are particularly enlightening? (sorry, I couldn’t help myself there)
Seriously, though, I’ve meditated before but it was difficult to determine whether any perceived benefits were really placebo or not.
Is there a formal system (not talking about the standard integers, I guess) whose provability oracle is strictly weaker than the halting oracle, but still uncomputable?
I did some reading and it looks like while there are some proofs of the existence of a dense set of intermediate Turing degrees, it’s a bit difficult to pin down definite problems of intermediate degree. I found one paper that postulates a couple of possibly intermediate degree problems.
This article (same author) talks about some of the surrounding issues and some the difficulties with proving the existence of an intermediate computational process.
Given those difficulties, it isn’t clear to me that intermediate proof oracles for formal systems exist and if they do it seems like that might be non-trivial, but I’m definitely not the best person to ask.
Subject: Problem Solving
Recommendation: Street-Fighting Mathematics The Art of Educated Guessing and Opportunistic Problem Solving
Reason: So, it has come to my attention that there is a freely available .pdf for the textbook for the MIT course Street Fighting Mathematics. It can be found here. I have only been reading it for a short while, but I would classify this text as something along the lines of ‘x-rationality for mathematics’. Considerations such as minimizing the number of steps to solution minimizes the chance for error are taken into account, which makes it very awesome.
in any event, I feel that this should be added to the list, maybe under problem solving? I’m not totally clear about that, it seems to be in a class of its own.
The book looks pretty interesting and that’s a nice story, but I’m not sure that this conclusion is much of a revelation. I’d be a bit more interested in why talking through an issue works when it does.
For instance, when I see
Part of the reason for the change was a historic conference held in Bermuda in 1996, and attended by many of the world’s leading biologists, including several of the leaders of the government-sponsored Human Genome Project.
and
The biologists in the room had enough clout that they convinced several major scientific grant agencies to make immediate data sharing a mandatory requirement of working on the human genome. Scientists who refused to share data would get no grant money to do research. This changed the game, and immediate sharing of human genetic data became the norm.
I think “Ok, so talking through something is important when most of the parties involved would be amenable to the issue, since they already have clout and don’t really need to fear rivals so much. When you happen to be part of a relatively powerful group that can make things happen via consensus, and it seems like there is an important issue you could garner consensus on, it would be good to gather up the group and have a chat.” This seems kind of trivial though.
Are we talking about non-rationalists, or non-x-rationalists? With Traditional rationalists (free thinkers, skeptics and the like), it seems that you could indeed phrase both of those issues are purely engineering problems with the strategy you are proposing, and expect some payoff.
With non-rationalists, however, it seems like the biggest part of the problems are these:
With strong AI, it is overcoming the tendency that people have to associate it with some foggy ideas about ‘consciousness’ they have cached, and thus conclude that it is ‘beyond’ the category of ‘mere engineering problem’.
With cryonics, a somewhat related issue occurs with the association of death with a ‘soul’ or an ‘afterlife’ and cryonics. When you get into the specifics of reviving the brain, similar issues about ‘consciousness’ can arise as well.
Going solely on what Eliezer has said about ‘exceeding your role models’, I would take that with a grain of salt. I’ve never met Eliezer, but although he comes off as extremely intelligent, judging by his writings and level of achievement (which are impressive) he still does not come off to me as, say, Von Neumann intelligent.
Eliezer’s writings have clarified my thoughts a great deal and given me a stronger sense of purpose. He is a very intelligent researcher and gifted explainer and evangelist, but I don’t take his word as Gospel, I take it as generally very good advice.
Winners Evoking Dangerous Recursively Improving Future Intelligences and Demigods
Anything for undergrads? It might be feasible to do a camp at the undergraduate level. Long term, doing an REU style program might be worth considering. NSF grants are available to non-profits and it may be worth at least looking into how SIAI might get a program funded. This would likely require some research, someone who is knowledgeable about grant writing and possibly some academic contacts. Other than that I’m not sure.
In addition, it might be beneficial to identify skill sets that are likely to be useful for SI research for the benefit of those who might be interested. What skills/specialized knowledge could SI use more of?
I stumbled over here from Scott Aaronson’s blog, which was recommended by a friend. Actually, LessWrong was also recommended, but unfortunately it took a while for me to make it over here.
As far as my descent in to rationality goes, I suppose I’ve always been curious and skeptical, but I never really gave much direction to my curiosity or my skepticism until the age of 17.
I always had intellectual interests. In 3rd and 4th grade I tought myself algebra. I ceased to pursue mathematics not too long after that due to the disappointment I felt towards the public school system’s treatment of mathematics.
After my foray into mathematics, I took a very strong interest in cosmology and astronomy. I still remember being 11 or 12 and first coming to realize that we are composed of highly organized cosmic dust. That was a powerful image to me at that time.
At this point in time I distinctly remember my father returning to the church after his mother and sister had passed away. The first church we went to was supposedly moderate. I was made to attend Sunday school there. I did not fare so well in sunday school. During the second session I attended the subject of evolution was brought up. Now, I had a fascination with prehistoric animals and had several books that explained evolution at a basic level accessible to young adults, so when the teacher challenged evolution and told me that the concept of God was not compatible with it, I told her that she must be wrong about God (this was really an appeal to authority, since I considered anyone who had written a book to be more authoritative than anyone who hadn’t). Well, she didn’t take that well and sent me to stand in the corner. My parents didn’t take well to that (both of them being fairly rational and open to science and my mom not being religious at all, but rather trying to support my dad). And so was borne my first religion-science conflict!
Once I entered high school, my artistic interests came to the foreground and pushed science and mathematics into the background. I developed my skill as a visual artists and as a guitarist. I studied music theory and color theory and played. It was enjoyable work and I took it to the point of obsession. My guitar playing especially, which I would practice for hours every night.
Eventually I decided that I wasn’t happy with making art, I wanted to explore something I felt was much deeper and more meaningful. Thus began a period of self reflection and a search for personal meaning. I decided that I wanted to explore my childhood interests, and so I began to study calculus and mechanics during my senior year of high school. It was also at that point that I read Crime and Punishment, Steppenwolfe, The Stranger and Beyond Good and Evil.
Soon I found my way to Kant and Russell. They in turn led me to Frege, Wittgenstein and Quine. My desire to understand myself soon extended to a desire to understand the world around me. Shortly after turning 18, I read Quine’s Methods of Logic and was surprised by how natural it felt to me (up until the undecidability part, which threw me through a loop at the time).
By that time, I had begun my major in mathematics. I took every (read every seemingly interesting) course I could to get as broad a view as I could as quickly as possible. This past year (my junior year of college) I took my first few graduate courses. The first was theory of computation. I had no prior experience with the material, everything was new. It was a somewhat transformative experience and I have to say that it was probably the most enjoyable class I’ve ever taken. I also took a graduate sequence in mathematical logic and learned the famed incompleteness theorems.
I am interested in fighting ignorance in myself and in others and I find that I like the premise of this blog. My current interests include Bayesian Probability (thanks to this site and Eliezer, and to some extent the friend who recommended it to me as well), the game of GO, physics (I am woefully ignorant of real physics, and have decided that I need to read up on it), mathematical logic, Fluid Concepts & Creative Analogies (Hofstadter), cognitive science, music, history and programming. It is not hard to get me interested in something, so the list is much more extensive than that and is highly subject to change.
Well, I feel like I’ve rambled up a storm here.
Typically you make a “sink” post with these sorts of polls.
ETA: BTW, I went for the paper. I tend to skim blogs and then skip to the comments. I think the comments make the information content on blogs much more powerful, however.
I think the real motivator that Runzel and Chatz provided is the opportunity for students to signal their dislike of Justin Beiber by donating.
I’m not sure, at the very least this could be inverted—they caused a situation where many groups of students would tend to pressure each other to donate in order to avoid the negative utility that would come from acting against their peer group.
If every student is signaling their dislike by donating, donating becomes the status quo. In that case, donating wouldn’t be a status raising action, but not signaling would deviate from the status quo, which is perceived as high risk so you’re willing to pay to avoid that. In the end, everyone but the groups for which the signaling is status neutral is at a net loss.
Many a man has cherished for years as his hobby some vague shadow of an idea, too meaningless to be positively false; he has, nevertheless, passionately loved it, has made it his companion by day and by night, and has given to it his strength and his life, leaving all other occupations for its sake, and in short has lived with it and for it, until it has become, as it were, flesh of his flesh and bone of his bone; and then he has waked up some bright morning to find it gone, clean vanished away like the beautiful Melusina of the fable, and the essence of his life gone with it. I have myself known such a man; and who can tell how many histories of circle-squarers, metaphysicians, astrologers, and what not, may not be told in the old German story?
Charles Sanders Peirce
Not to mention a massive underestimation of intermediate positions, e.g. the doubting faithful, agnostics, people with consciously chosen, reasonable epistemology etc. This sets that number to 0. I’ve met plenty of more liberal theists that didn’t assert 100% certainty.
I’ve asked the same thing, both of myself (I’ve got it, though I make a conscious effort to suppress it) and others. My conclusion? It is knee-jerk feeling of embarrassment.
Were you perhaps home schooled or brought up in some other non-traditional environment? I ask because I went to public school, and I can tell you that it isn’t only in the adult world that people don’t like to be corrected. Children routinely make fun of or prod other children when they make mistakes, and at the same time they might shrug off a correction or even make fun of you for making it or for knowing the fact.
There is a fairly good TED talk related to this topic here.
I’ve thought of a few comments:
1) If they are reset every hour of subjective time that would put some serious bounds on how much information they could usefully pass on, especially if it is in the form of a virtual book. Maybe if you rewrote the component of the upload corresponding to memory this would work, but then why bother to reset? Is it to avoid boredom? I suppose you could only rewrite a restricted part of the memory of the upload. Why not try to tweak the upload to alleviate whatever issues you are anticipating (make it not get bored ect.)?
2) Assuming this upload is actually smart enough to make any progress in taking over the world, how do you guard against them deciding that they don’t like being reset, and cleverly passing on a plan to eventually prevent you from resetting them? Even Gandhi might not appreciate being put in this sort of scenario.
3) I’m a little unsure of the effect that isolating it from all of the intellectual community might have on its effectiveness as a researcher, it seems like a large part of academic effectiveness comes from the availability of multiple perspectives. Maybe it would make more sense to try to simulate a small community of scholars rather than just one?
I’ll start with the obvious: Math 18.05, Introduction to Probability and Statistics.
It seems obvious that street fighting mathematics would be highly appropriate.
The epistemic hygiene is a huge factor for me. It is quite difficult to find a community that is nearly this active while retaining such high standards for content. Most of the online communities I’ve been involved with had a small pocket of individuals with relatively high standards, but I’ve never found such a degree of uniform quality as I’ve found here.