I think the most important sentence for LWers was: ”...if it is not an American Singularity it is going to be a Chinese Singularity...” (David Chalmers). He said that he has been talking about it at a military academy and that thought arose in the end.
Politics might be a mind-killer but now that the Singularity gets front-page coverage from popular magazines, acceptance within the academics and military attention, it is only a question of time until this topic will be discussed by politicians. The problem is, what does this mean for the SIAI? Can the SIAI avoid political struggle once people like Sarah Palin take it seriously? Will they allow the SIAI to keep working unsupervised, or at all? Can the SIAI compete with DARPA? Is the SIAI ready for large scale attacks by foreign spies, for example Chinese cyber attacks?
The problem is, what does this mean for the SIAI? Can the SIAI avoid political struggle once people like Sarah Palin take it seriously?
This prompts the age old question “Could Eliezer advocate FAI with enough credibility that even writing more fanfiction couldn’t make him seem silly?” (I hope not!)
I think the most important sentence for LWers was: “If it is not an American Singularity it is going to be a Chinese Singularity” (David Chalmers). He said that he has been talking about it at a military academy.
I can’t say either of those options appeals to me especially. How is FAI research going in, say, Canada?
It is not that he suggested that, it is taken out of context. He said that was the thought that arose in the end while discussing the topic of superhuman AI at a military academy. And I wanted to highlight that such thoughts might be the likely outcome if the idea gets more popular. It will be we vs. them. Politicians and the military might actually encouraging building fooming AI as fast as possible so that the Chinese don’t take over.
A fairly short “bite”. Brief summary of contents: A fair bit about uploading, brain emulation and the simulation hypothesis. Are uploads conscious? Are they “really” you?
Blurb reads: “Philosopher David Chalmers discusses the philosophical implications of this imaginable situation with Nigel Warburton in this episode of the Philosophy Bites podcast.”
I listened to it, but admittedly spaced out a little towards the end. I doubt there’s anything in there which will be new to people here, but it might be of interest for tracking what’s being said about the Singularity.
It at least mentioned Friendliness problems, though not the problem of designers meaning well but making catastrophic mistakes about the details.
There was one bit which was confirmation of something I’ve been talking about—Chalmers gave a talk at West Point, and the question of whether the military would try to prevent the creation of an AI because it could be such a security threat.
By implication, I think what’s meant is something like lobbying to prevent the US government from funding AI projects rather than anything more dramatic.
Anyway, one of the cadets said that even so, it was important for the US to do it, because otherwise it would be done by the Chinese.
Annoying stereotype time: this universe may have been created by a “smelly hacker” in the next universe up. It’s at least as likely that creating universes is a respectable mainstream hobby. It’s arguably more likely, since more of something gets done if it’s mainstream and respectable.
However, let’s kick it around. There’s certainly more people watching football (mainstream and respectable) than tatting (respectable but not mainstream). It’s harder to be sure about activities that are not respectable, though I’d bet there are more people watching football than engaging in zoophilia, or even than watching zoophilia.
Admittedly, it’s hard to be sure of choosing the right examples. Would drinking during Prohibition be mainstream but not respectable, or mainstream and respectable? Respectable unless you’re in a non-drinking culture?
Depending on how we cash out “mainstream” it could build “popular” right into it, so I’m not sure how to evaluate that half of the question non-tautologically. Respectability seems slippery. Is, say, masturbation “respectable”? Depends on the subculture, maybe, but overall I suspect lots of it gets done even in places where it’s not considered so. Is picking one’s nose “respectable”? Would more of it get done if it were?
Is picking one’s nose “respectable”? Would more of it get done if it were?
I think so, especially if it were respectable enough to be done in public.
I think “mainstream” isn’t quite the same as “popular”—it’s got a little edge of “we congratulate ourselves for being normal”, which is not quite the same thing as “respectable”, which I’d say means “confers status in a way which implies social stability”.
This started with the image of the “smelly hacker”—someone who is low status and a member of a not especially numerous group.
Compare this to the image of water-color painting for respectable young Victorian women. (It would be so embarrassing if creating universes like ours was highly derivative art.) I’m sure there were a lot of those water color paintings.
It’s interesting that it’s hard to come up with examples of even mildly widespread creative hobbies. Maybe quilting and whittling in earlier times?
Or maybe entities buy universes, tweak slightly, and just watch them evolve. It could be like owning an aquarium.
A lot of this is about costs of entry. Picking one’s nose has a very low cost of entry, while the image of the smelly hacker is a person who does things which have a high cost of entry which are generally not considered to be worth doing.
What simulations would cost in the next universe up is a question that I’d say is hard to address since we don’t know what the physics would be like. Our universe could be anything from very expensive bright flower of the culture like the first moon shot to the equivalent of drumming one’s fingers.
Thanks for asking. The question has gone off in some interesting directions.
Do you have one sentence worth of commentary with respect to how worthwhile the podcast episode may be?
I think the most important sentence for LWers was: ”...if it is not an American Singularity it is going to be a Chinese Singularity...” (David Chalmers). He said that he has been talking about it at a military academy and that thought arose in the end.
Politics might be a mind-killer but now that the Singularity gets front-page coverage from popular magazines, acceptance within the academics and military attention, it is only a question of time until this topic will be discussed by politicians. The problem is, what does this mean for the SIAI? Can the SIAI avoid political struggle once people like Sarah Palin take it seriously? Will they allow the SIAI to keep working unsupervised, or at all? Can the SIAI compete with DARPA? Is the SIAI ready for large scale attacks by foreign spies, for example Chinese cyber attacks?
This prompts the age old question “Could Eliezer advocate FAI with enough credibility that even writing more fanfiction couldn’t make him seem silly?” (I hope not!)
I can’t say either of those options appeals to me especially. How is FAI research going in, say, Canada?
It is not that he suggested that, it is taken out of context. He said that was the thought that arose in the end while discussing the topic of superhuman AI at a military academy. And I wanted to highlight that such thoughts might be the likely outcome if the idea gets more popular. It will be we vs. them. Politicians and the military might actually encouraging building fooming AI as fast as possible so that the Chinese don’t take over.
DARPA do fund a fair bit of artificial intelligence research already:
http://www.wired.com/dangerroom/2009/05/darpa-heat-energy-brains-now-make-us-some/
http://www.switched.com/2010/03/30/darpas-guard-dog-aims-to-replace-human-intelligence/
Then there’s IARPA:
Uh huh...
A fairly short “bite”. Brief summary of contents: A fair bit about uploading, brain emulation and the simulation hypothesis. Are uploads conscious? Are they “really” you?
Blurb reads: “Philosopher David Chalmers discusses the philosophical implications of this imaginable situation with Nigel Warburton in this episode of the Philosophy Bites podcast.”
I listened to it, but admittedly spaced out a little towards the end. I doubt there’s anything in there which will be new to people here, but it might be of interest for tracking what’s being said about the Singularity.
It at least mentioned Friendliness problems, though not the problem of designers meaning well but making catastrophic mistakes about the details.
There was one bit which was confirmation of something I’ve been talking about—Chalmers gave a talk at West Point, and the question of whether the military would try to prevent the creation of an AI because it could be such a security threat.
By implication, I think what’s meant is something like lobbying to prevent the US government from funding AI projects rather than anything more dramatic.
Anyway, one of the cadets said that even so, it was important for the US to do it, because otherwise it would be done by the Chinese.
Annoying stereotype time: this universe may have been created by a “smelly hacker” in the next universe up. It’s at least as likely that creating universes is a respectable mainstream hobby. It’s arguably more likely, since more of something gets done if it’s mainstream and respectable.
This seemed sensible on first inspection, but now that I think about it again I’m not so sure. What makes you say this?
It seemed sensible on first inspection.
However, let’s kick it around. There’s certainly more people watching football (mainstream and respectable) than tatting (respectable but not mainstream). It’s harder to be sure about activities that are not respectable, though I’d bet there are more people watching football than engaging in zoophilia, or even than watching zoophilia.
Admittedly, it’s hard to be sure of choosing the right examples. Would drinking during Prohibition be mainstream but not respectable, or mainstream and respectable? Respectable unless you’re in a non-drinking culture?
What’s your take on the question?
Depending on how we cash out “mainstream” it could build “popular” right into it, so I’m not sure how to evaluate that half of the question non-tautologically. Respectability seems slippery. Is, say, masturbation “respectable”? Depends on the subculture, maybe, but overall I suspect lots of it gets done even in places where it’s not considered so. Is picking one’s nose “respectable”? Would more of it get done if it were?
I think so, especially if it were respectable enough to be done in public.
I think “mainstream” isn’t quite the same as “popular”—it’s got a little edge of “we congratulate ourselves for being normal”, which is not quite the same thing as “respectable”, which I’d say means “confers status in a way which implies social stability”.
This started with the image of the “smelly hacker”—someone who is low status and a member of a not especially numerous group.
Compare this to the image of water-color painting for respectable young Victorian women. (It would be so embarrassing if creating universes like ours was highly derivative art.) I’m sure there were a lot of those water color paintings.
It’s interesting that it’s hard to come up with examples of even mildly widespread creative hobbies. Maybe quilting and whittling in earlier times?
Or maybe entities buy universes, tweak slightly, and just watch them evolve. It could be like owning an aquarium.
A lot of this is about costs of entry. Picking one’s nose has a very low cost of entry, while the image of the smelly hacker is a person who does things which have a high cost of entry which are generally not considered to be worth doing.
What simulations would cost in the next universe up is a question that I’d say is hard to address since we don’t know what the physics would be like. Our universe could be anything from very expensive bright flower of the culture like the first moon shot to the equivalent of drumming one’s fingers.
Thanks for asking. The question has gone off in some interesting directions.