I agree with both of your mentioned drawbacks and would add a third: If someone actually discovered that it’s not just a human-level AI but a self-improving AGI with a mission (or if someone from the team leaked that information) the public backlash would be absolutely fatal. And then there would also be the added problem of how to introduce the AGI in a fashion that’s not essentially an unconsenting takeover of planet earth.
As for my participation in AI-research you don’t need to worry. I can hardly code a website and have no intention of learning it or participating directly in AI development. I’m coming from a psychological background, which is probably why I’m unusually concerned about the social repercussions of the self-improving AGI meme. (“Give him a hammer and suddenly everything looks like a nail” as the saying goes. Conversely, some of the technically inclined people here may not even realize, that there may be much more to pulling off AGI than the strictly technical aspects.)
You are aware, that currently about 40% of Americans believe in full-blown creationism and another 40% that god guided the process of evolution with us in mind, right? Ten years ago almost 20% believed the sun orbits our earth and I wouldn’t be terribly surprised if essentially nothing about that state of affairs has changed in the meantime...
Even if such beliefs will somewhat recede in the upcoming 30 years, you should be painfully aware that under such conditions you will never ever get public consent for the development of self-improving AGI. At least not from primal unenhanced brains. So if you ever put our idea of the future up for vote, we will lose for sure. And that’s just America, the rest of the planet of the apes -including my beloved Europe- won’t be amused about our futuristic plans either, and even less amused if some other nation or an American corporate giant like google or IBM tried to pull off something like this in a solo attempt.
So I’ll just call it how I see it: Do you want to make self-improving AGI a reality? Then we’ll have to find a way to make it happen without involving public opinion in this decision. They’ll never consent, no matter how honest and detailed and soulful you pitch our awesome cause. I hope no one here is naive enough to expect that we can pull off self-improving AI without some kind of clash of interests. Given this reality, and considering your objections (and more) my impression is that keeping a low profile (not now, but eventually) will be in our best interest. The best possible thing that I can imagine happening, is if this endeavor becomes a scientific project backed by numerous nations—this will largely prevent the perception and possibility that the developed AGI could be used as a potential weapon, it would allow to bring clever minds from all over the globe into the boat (countering your second point) and last but not least this approach may mitigate public distrust towards a manageable point. (“Every other nation does it too!”).
The challenge is obviously how (in a decade or whatever) you could pitch our self-improving AI vision to people who have built their careers on sucking up to the common denominator and who would probably become obsolete through the very arrival of this technology. We need great salesmen ;)
PS: Don’t think I’m not sympathetic to the idea that we should communicate our cause honestly and openly all the way until the bitter (sweet?) end… yet I don’t believe it will work that way. The majority of people do not share our vision and they wouldn’t vote for it. Ever.
So I’ll just call it how I see it: Do you want to make self-improving AGI a reality? Then we’ll have to find a way to make it happen without involving public opinion in this decision. They’ll never consent, no matter how honest and detailed and soulful you pitch our awesome cause.
Really? That’s not the impression I got from those numbers at all. To me, it sounds less like the public is adamantly resolved to stick with those entrenched ideas, and more like most people will believe all sorts of insane bullshit of you can spin a plausible-sounding explanation of how they might benefit by believing in it, and if you persist long enough. Do you really think the vote would be a one-time thing?
To me, it sounds less like the public is adamantly resolved to stick with those entrenched ideas, and more like most people will believe all sorts of insane bullshit of you can spin a plausible-sounding explanation of how they might benefit by believing in it, and if you persist long enough.
There may be something to that perspective, but I think it is unrealistic to expect we could change enough people’s minds in so short of a time-frame. There’s a lot of people out there. Religions have had thousands of years to adapt themselves in such a way, that they reinforce and play into people’s innate superstitions and psychological desires. In turn, religions also shaped people’s culture and until very recently they played the major role in the “nurture” side of the “nature and nurture” make-up of people. Competing with religion on our own terms (rationality) simply won’t work so well with the majority of people.
Understanding our AGI “message” requires various quantum leaps in thinking and rationality. These insights implicitly and explicitly challenge most innate intuitions about reality and humanity that people currently hold on to. I’m not saying there won’t be many people we could be able to persuade without a thorough education in these matters, but because in contrast to religion our “worldview” doesn’t tell people what deep down they would like to hear and believe, we’re less attractive to those who just can’t be arsed into rationality. Which are a lot o people.
In conclusion, I’ll sum up my basic point in another light yet again: I think I’m not confronting us with a false dichotomy, when I say that there are essentially only two possibilities when it comes to introducing AGI into the lives of people:
EITHER we’re willing to adhere to public consent along current democratic principles. This would entail, that we massively concern ourselves with public opinion and make a commitment to not unleash AGI, unless the absolute majority of all citizens on this planet (or those who we consider to meet the criteria of valid consent) approve of our plan.
OR, we take the attitude that people who do not meet a certain standard of rationality have no business in shaping humanity’s future and we become comfortable with deciding over their heads/on their behalf. This second option certainly does not light up any applause lights for believers in democracy, but I believe among lesswrongers this may not be that much of an unpopular notion.
You can’t have it both ways: Either you commit yourself to the insane effort of making sure, that the issue of AGI gets decided in a democratic and “fair” fashion, or you aim at some “morally” “lower” standard and are okay with not everyone getting their say when it comes to this issue. As far as I’m concerned you know my current preference, which I favor because I find the alternative to be completely unrealistic and I’m vastly more committed to rationality than I am to the idea that undiscriminating democracy is the gold standard of decision-making.
What about representative democracy? Any given community sends off a few of it’s cleverest individuals with a mandate to either directly argue for that community’s interests, or to select another tier of representatives who will do so. Nobody feels completely excluded, but only a tiny fraction of the overall population actually needs to be educated and persuaded on the issues.
How is it representative, if only the cleverest individuals are chosen? That would rather be elitism. If actually only the most rational people with herculean minds would decide, they should theoretically unanimously agree to either do it or not do it anyway, based on a sound probability-evaluation and shared premises based in reality that they all agree on.
If those “representative” individuals were democratically determined by vote, then these people most certainly won’t be the most intelligent and rational people, but those best at rhetorically convincing others and sucking up to them by exploiting their psychological shortcomings. They would simply be politicians like the ones we have nowadays.
So in a way we’re where we started. If people don’t decide for themselves, they’ll simply vote for someone who represents their (or provides them with a new) uninformed opinion. Whoever wins such an election will not be the most rational person, that’s for sure (remember when America voted twice for an insane cowboy?)
While representative democracy is certainly more practical than the alternatives, I doubt the outcome would be all that better. If we want the most rational and intelligent people to make this decision, then these individuals couldn’t be chosen by vote but only by another “elitist” group. I don’t know how the public would react to that—I suppose they would not be flattered.
I’m not saying it would be a better system overall, just that a relatively small group of politicians would be comparatively easier for us to educate and/or bribe.
I’m still puzzled though which approach would be better… involving and educating the politicians (there are many who wouldn’t understand) or trying to keep them out as long as possible to avoid confrontations and constraints? I already remarked somewhere, that I would find some kind of international effort towards AGI development very preferable, something comparable to CERN would be brilliant. Such a team could first work towards human-level AI and then one-up themselves with self-improving AGI once they gained some trust for their competence.
In other words, perhaps advertising and reaching the “low-hanging fruit” of human-level AI plus reaping the amazing benefits of such a breakthrough will raise public and political trust in them, as opposed to some “suspicious” corporation or national institute that suddenly builds potential “weapons” of mass destruction.
So I’ll just call it how I see it: Do you want to make self-improving AGI a reality? Then we’ll have to find a way to make it happen without involving public opinion in this decision.
Well, I’m not at all convinced that substantially self-improving AGI can exist (that is, that will self-improve at such a rate as to quickly gain near complete control of its light cone or something like that). I assign only a small chance to the likelyhood that the first AGI will go foom. Also, if I’ve learned one thing from LW it is that such AI could plausibly be really bad. So I’d rather take a risk averse strategy if it at all possible.
So if you ever put our idea of the future up for vote, we will lose for sure.
Rule number 1 of voting: it’s done after a thorough debate where every single party has said everything they wanted to say. Not to generalize from fictional evidence, but “Twelve Angry Man” has shown us a pretty good caricature of the dramatic changes that can happen if you prolong the debate just a little bit longer before voting. Which is why educating the public and letting the ideas circulate is so crucial.
They’ll never consent, no matter how honest and detailed and soulful you pitch our awesome cause. The majority of people do not share our vision and they wouldn’t vote for it. Ever.
You haven’t justified this. What does believeing God guided Evolution have to do with making plans to build a self-improving artificial intelligence?
More importantly, isn’t it better that they know about it, and forbid it or put it under extremely intense scrutiny, rather than they not know about it, and some group developing it and obscurity and botching it?
I think you’re highly delusional about how malleable people’s opinions really are… Are you aware of what’s going on in politics and the religious sphere? As if just talking really thoroughly about AGI and appealing to rationality is going to get the majority of people from all over the world on our side. Are you serious?
The point I made about creationism wasn’t just that most people who believe in god probably won’t want to see one being built, but that you cannot change people’s opinions easily. Even completely ridiculous and unworldly ideas like creationism have hardly budged an inch in the last decade—they are rationalityproof. If you really thoroughly explained to people what this self-improving AGI was good for and how powerful it could really become… they’d totally lose it. They won’t welcome “our robot overlords”, regardless of how nice you make the resulting utopias sound. People fear the unknown and on a gut-level they will immediately reject our idea and rationalize in the blink of an eye why we’re wrong, and crazy, and have to be stopped.
I’m all for thoroughly educating people about rationality (you’ve read my suggestion in the other topic), but seriously getting the majority of people behind us? Sorry, but my psychological model of how people and masses behave tells me that this will never happen. At least not without brain-augmentation, and even then a global 50% +1 vote seems quite unlikely to me.
Would it be better if people knew in detail about self-improving AGI and could objectively discuss this matter in order to rationally make a decision and responsibly vote on whether or not it should be developed? Hell yeah I’d love that! I’d also love to ride on a flying pig but that’s not gonna happen either.
I’d prefer it if you used “mistaken” rather than “delusional”, thank you very much. Ascribing opposing opinions to madness usually signals weakness in your own stance.
Are you aware of what’s going on in politics and the religious sphere?
Quite, see my next paragraph.
Are you serious?
I am positive that talking about it publicly and rationally will bring humanity, eventually, to the side of reason, whomsoever it may lie with. Maybe as a US citizen you see things differently (although from here I see education improving somewhat, steps being taken towards giving citizens the minimum necessary services, political awareness developing, racial and sexual and gender issues being slowly abolished...)
But as a resident of Europe and the Middle East, I see religion and partisanship shriveling into a husk, intelligence and culture extending and growing, triumphant, as they never have before, and the citizenry reclaiming power over the aloof governments, and over their futures.
Humanism wins. And rationalism cannot lose, on the long term, because, as its name indicates, it is the art of being right.
creationism has hardly budged an inch in the last decade
That’s not what I heard at all. Creationism only became acknowledged as a problem recently. Which means it was secure before, and it is now being challenged, and singing it’s swan song. Lack of visible, spectacular budging doesn’t mean that it isn’t crumbling from the inside. And it’s really a problem that is endemic to the USA: across the pond, virtually no one believes in Creationism. I suspect this has something to do with the education of the masses, which is very overlooked in the USA. Once the US society will feel the need to raise their own education level, for any economic reasons, the problems derived from ignorance will just extinguish themselves by sheer lack of combustible.
People fear the unknown and on a gut-level they will immediately reject our idea and rationalize in the blink of an eye why we’re wrong, and crazy, and have to be stopped.
That’s why the improvement of public, mass education, and the spreading of our Art must be a priority, if not our number one priority. That’s why I said “explaining things thoroughly: by that I mean raising the level of awareness of the general public*.
Here in Spain, France, the UK, the majority of people are Atheists. In the USSR virtually everyone were atheists. Beliefs are extremely malleable. By Raising The Sanity Waterline, we’ll make it so that they are only malleable through empirical evidence, and, if we do it right, people won’t even notice ).
I’d also love to ride on a flying pig but that’s not gonna happen either.
You believe in Transhuman-level cybernetics and brain expansions and you don’t believe we can make pigs fly and carry people on their backs?
I’d prefer it if you used “mistaken” rather than “delusional”
Funny, on my second read-through I thought about editing it, but then my mind went “whatever, a healthy ego can probably take it”. No hurt feelings I hope.
I’m also living in the EU, but I’m very aware and constantly following what’s going on in the US, because their erratic development concerns me. As far as evolution and creationism goes, I’m drawing my statistics from the Gallup polls: in the last 30 years creationism went down to 40% from 45%, “evolution through divine intervention” remained stagnant at 40% and “plain natural selection” went up about 5% to 15%.
There is a positive movement here, but I don’t see how in another 30 years the collapse of religion will be imminent. And that’s just the US, to a resident of the Middle East I shouldn’t need to point out, that there are plenty of countries out there much more religious than the US. (Essentially all of them, apart from a few developed countries—and even those countries have usually only around 20% confirmed atheists. Many don’t visit the church, but they’re still holding on to a mountain of superstitious garbage → http://en.wikipedia.org/wiki/Demographics_of_atheism#Europe )
You’re of course also right that here in Europe there’s hardly any creationists—unlike in the US it’s just too damaging to one’s reputation, so people conveniently adapt their views. I doubt however, that this has all that much to do with the quality of education, and a lot more with cultural attitudes towards religion. As far as Europe goes, I can imagine that the church in the biggest countries (France, Germany, Spain, UK...) will be essentially dead in another 30 years, but what has that to say about people’s ability to make rational decisions? There’s a lot more to rationality than not believing in obvious bs.
There’s gonna be close to 9 billion people in 30 years and you think we -a tiny speck of nerds like LW- could hope to reach out and educate a sizable portion (almost half no less) of the world’s people in the art of rationality and in scientific understanding, so we can put AGI up for vote? And it’s not just simply educating them mind you—in a struggle of memes the application of rationality would require them to throw out just about all of their cherished beliefs about life and the world! And you also seem to be forgetting, that the average LW-IQ may perhaps lie somewhere around 120, and that most people aren’t actually all that clever.
I’m an optimist, but I’m afraid without brain-augmentation something like this just isn’t in the realm of possibilities. There’s no way how you could polish our message to a point where it could stick for so many different people.
You believe in Transhuman-level cybernetics and brain expansions and you don’t believe we can make pigs fly and carry people on their backs?
Damn it! I knew someone would say that and ruin my rhetoric.
As a resident of the Middle East, I can tell you that mentalities are changing fast. Regardless, the attitude towards Science isn’t the same as Christians, since they don’t feel threatened by it, believing that the Qur’an not only isn’t in conflict with science, but actually anticipated some discoveries. They also reclaim the development of modern scientific research as a proud heritage they are enthusiastic to live up to again, and believe researchers should be left alone to investigate, no matter how outrageous the stuff they come up with is (if i remember well think there’s even a command in the Qran specifically to that effect).
As for countries that have been converted to major religions by colonialism, I have a strong feelings that they would actually convert to whatever looks coolest, most Western and most high-status-signalling. We just need to be about 20% cooler than everyone else. Seems manageable.
you also seem to be forgetting, that the average LW IQ may lie somewhere around 120 and that most people aren’t actually all that clever
We should be able to teach rationality to anyone capable of deliberative thought. That is, anyone with an IQ over 70. that the original developers and vanguard be more fast-learning than average is not surprising at all.
Our stuff is simpler, less confusing, far clearer, and far more useful, than anything any religion can teach. I think people could definitely be attracted to our lack of bullshit, if we sell it right.
Checks his numbers Forgive me. I should have said the majority of young people (below 30) who, for our uses and purposes, are those who count, and the target demographic. It has come to the point that self-declared Christian kids get bullied and insulted [which is definitely wrong and stupid and not a very good sign that the Sanity Waterline was raised much].
Then again, I have this rule of thumb that I don’t count people who don’t attend church as believers, and automatically lump them into the “atheist in the making” category, a process that is definitely not legitimate nor fair. I sincerely apologize for this, and retract the relevant bits.
Now let’s see. For one thing
Statistics on atheism are often difficult to represent accurately for a variety of reasons. Atheism is a position compatible with other forms of identity. Some atheists also consider themselves Agnostic, Buddhist, Jains, Taoist or hold other related philosophical beliefs. Therefore, given limited poll options, some may use other terms to describe their identity. Some politically motivated organizations that report or gather population statistics may, intentionally or unintentionally, misrepresent atheists. Survey designs may bias results due to the nature of elements such as the wording of questions and the available response options. Also, many atheists, particularly former Catholics and former Mormons, are still counted as Christians in church rosters, although surveys generally ask samples of the population and do not look in church rosters. Other Christians believe that “once a person is [truly] saved, that person is always saved”, a doctrine known as eternal security.[5] Statistics are generally collected on the assumption that religion is a categorical variable. Instruments have been designed to measure attitudes toward religion, including one that was used by L. L. Thurstone. This may be a particularly important consideration among people who have neutral attitudes, as it is more likely prevailing social norms will influence the responses of such people on survey questions which effectively force respondents to categorize themselves either as belonging to a particular religion or belonging to no religion. A negative perception of atheists and pressure from family and peers may also cause some atheists to disassociate themselves from atheism. Misunderstanding of the term may also be a reason some label themselves differently.
The fact that Jedi outnumber Jews in the UK should be a sign that people don’t take that part of the polls very seriously.
That said
Several studies have found Sweden to be one of the most atheist countries in the world. 23% of Swedish citizens responded that “they believe there is a God”, whereas 53% answered that “they believe there is some sort of spirit or life force” and 23% that “they do not believe there is any sort of spirit, God, or life force”. This, according to the survey, would make Swedes the third least religious people in the 27-member European Union, after Estonia and the Czech Republic. In 2001, the Czech Statistical Office provided census information on the ten million people in the Czech Republic. 59% had no religion, 32.2% were religious, and 8.8% did not answer.[16]
A 2006 survey in the Norwegian newspaper Aftenposten (on February 17), saw 1,006 inhabitants of Norway answering the question “What do you believe in?”. 29% answered “I believe in a god or deity,” 23% answered “I believe in a higher power without being certain of what,” 26% answered “I don’t believe in God or higher powers.” and 22% answered “I am in doubt.” Still, some 85% of the population are members of the Norwegian state’s official Lutheran Protestant church. This may result from Norwegians being registered into the church at birth, yet having to intentionally unregister after becoming adults.
In France, about 12% of the population reportedly attends religious services more than once per month. In a 2003 poll 54% of those polled in France identified themselves as “faithful,” 33% as atheist, 14% as agnostic, and 26% as “indifferent.”[17] According to a different poll, 32% declared themselves atheists, and an additional 32% declared themselves agnostic.[18]
In Spain, 81.7% are believers, 11% are non-believers and 6% are atheists (according to the 2005 poll of the public Centro de Investigaciones Sociológicas).[19]
This last bit i found particularly troubling because I do not recall metting a single person, in all my time in Spain, who declared themselves a Christian except in name only (as in, embarassingly confessing they only got baptized or went to Communion to please the grandparents). Some entertained some vague fuzziness, but simply telling them a little about “belief in belief” and some reductionist notions has been enough to throw them in serious doubt. I may very well be mistaken, but my perception is that they are really ripe for the taking, and only need to hear the right words.
My perception as a young Arab-European is that the trend is overwhelmingly in the direction of faithlessness, and that it is an accelerating process with no stopping force in sight.
I agree with both of your mentioned drawbacks and would add a third: If someone actually discovered that it’s not just a human-level AI but a self-improving AGI with a mission (or if someone from the team leaked that information) the public backlash would be absolutely fatal. And then there would also be the added problem of how to introduce the AGI in a fashion that’s not essentially an unconsenting takeover of planet earth.
As for my participation in AI-research you don’t need to worry. I can hardly code a website and have no intention of learning it or participating directly in AI development. I’m coming from a psychological background, which is probably why I’m unusually concerned about the social repercussions of the self-improving AGI meme. (“Give him a hammer and suddenly everything looks like a nail” as the saying goes. Conversely, some of the technically inclined people here may not even realize, that there may be much more to pulling off AGI than the strictly technical aspects.)
You are aware, that currently about 40% of Americans believe in full-blown creationism and another 40% that god guided the process of evolution with us in mind, right? Ten years ago almost 20% believed the sun orbits our earth and I wouldn’t be terribly surprised if essentially nothing about that state of affairs has changed in the meantime...
Even if such beliefs will somewhat recede in the upcoming 30 years, you should be painfully aware that under such conditions you will never ever get public consent for the development of self-improving AGI. At least not from primal unenhanced brains. So if you ever put our idea of the future up for vote, we will lose for sure. And that’s just America, the rest of the planet of the apes -including my beloved Europe- won’t be amused about our futuristic plans either, and even less amused if some other nation or an American corporate giant like google or IBM tried to pull off something like this in a solo attempt.
So I’ll just call it how I see it: Do you want to make self-improving AGI a reality? Then we’ll have to find a way to make it happen without involving public opinion in this decision. They’ll never consent, no matter how honest and detailed and soulful you pitch our awesome cause. I hope no one here is naive enough to expect that we can pull off self-improving AI without some kind of clash of interests. Given this reality, and considering your objections (and more) my impression is that keeping a low profile (not now, but eventually) will be in our best interest. The best possible thing that I can imagine happening, is if this endeavor becomes a scientific project backed by numerous nations—this will largely prevent the perception and possibility that the developed AGI could be used as a potential weapon, it would allow to bring clever minds from all over the globe into the boat (countering your second point) and last but not least this approach may mitigate public distrust towards a manageable point. (“Every other nation does it too!”).
The challenge is obviously how (in a decade or whatever) you could pitch our self-improving AI vision to people who have built their careers on sucking up to the common denominator and who would probably become obsolete through the very arrival of this technology. We need great salesmen ;)
PS: Don’t think I’m not sympathetic to the idea that we should communicate our cause honestly and openly all the way until the bitter (sweet?) end… yet I don’t believe it will work that way. The majority of people do not share our vision and they wouldn’t vote for it. Ever.
Really? That’s not the impression I got from those numbers at all. To me, it sounds less like the public is adamantly resolved to stick with those entrenched ideas, and more like most people will believe all sorts of insane bullshit of you can spin a plausible-sounding explanation of how they might benefit by believing in it, and if you persist long enough. Do you really think the vote would be a one-time thing?
There may be something to that perspective, but I think it is unrealistic to expect we could change enough people’s minds in so short of a time-frame. There’s a lot of people out there. Religions have had thousands of years to adapt themselves in such a way, that they reinforce and play into people’s innate superstitions and psychological desires. In turn, religions also shaped people’s culture and until very recently they played the major role in the “nurture” side of the “nature and nurture” make-up of people. Competing with religion on our own terms (rationality) simply won’t work so well with the majority of people.
Understanding our AGI “message” requires various quantum leaps in thinking and rationality. These insights implicitly and explicitly challenge most innate intuitions about reality and humanity that people currently hold on to. I’m not saying there won’t be many people we could be able to persuade without a thorough education in these matters, but because in contrast to religion our “worldview” doesn’t tell people what deep down they would like to hear and believe, we’re less attractive to those who just can’t be arsed into rationality. Which are a lot o people.
In conclusion, I’ll sum up my basic point in another light yet again: I think I’m not confronting us with a false dichotomy, when I say that there are essentially only two possibilities when it comes to introducing AGI into the lives of people:
EITHER we’re willing to adhere to public consent along current democratic principles. This would entail, that we massively concern ourselves with public opinion and make a commitment to not unleash AGI, unless the absolute majority of all citizens on this planet (or those who we consider to meet the criteria of valid consent) approve of our plan.
OR, we take the attitude that people who do not meet a certain standard of rationality have no business in shaping humanity’s future and we become comfortable with deciding over their heads/on their behalf. This second option certainly does not light up any applause lights for believers in democracy, but I believe among lesswrongers this may not be that much of an unpopular notion.
You can’t have it both ways: Either you commit yourself to the insane effort of making sure, that the issue of AGI gets decided in a democratic and “fair” fashion, or you aim at some “morally” “lower” standard and are okay with not everyone getting their say when it comes to this issue. As far as I’m concerned you know my current preference, which I favor because I find the alternative to be completely unrealistic and I’m vastly more committed to rationality than I am to the idea that undiscriminating democracy is the gold standard of decision-making.
What about representative democracy? Any given community sends off a few of it’s cleverest individuals with a mandate to either directly argue for that community’s interests, or to select another tier of representatives who will do so. Nobody feels completely excluded, but only a tiny fraction of the overall population actually needs to be educated and persuaded on the issues.
How is it representative, if only the cleverest individuals are chosen? That would rather be elitism. If actually only the most rational people with herculean minds would decide, they should theoretically unanimously agree to either do it or not do it anyway, based on a sound probability-evaluation and shared premises based in reality that they all agree on.
If those “representative” individuals were democratically determined by vote, then these people most certainly won’t be the most intelligent and rational people, but those best at rhetorically convincing others and sucking up to them by exploiting their psychological shortcomings. They would simply be politicians like the ones we have nowadays.
So in a way we’re where we started. If people don’t decide for themselves, they’ll simply vote for someone who represents their (or provides them with a new) uninformed opinion. Whoever wins such an election will not be the most rational person, that’s for sure (remember when America voted twice for an insane cowboy?)
While representative democracy is certainly more practical than the alternatives, I doubt the outcome would be all that better. If we want the most rational and intelligent people to make this decision, then these individuals couldn’t be chosen by vote but only by another “elitist” group. I don’t know how the public would react to that—I suppose they would not be flattered.
I’m not saying it would be a better system overall, just that a relatively small group of politicians would be comparatively easier for us to educate and/or bribe.
Yes, that is true.
I’m still puzzled though which approach would be better… involving and educating the politicians (there are many who wouldn’t understand) or trying to keep them out as long as possible to avoid confrontations and constraints? I already remarked somewhere, that I would find some kind of international effort towards AGI development very preferable, something comparable to CERN would be brilliant. Such a team could first work towards human-level AI and then one-up themselves with self-improving AGI once they gained some trust for their competence.
In other words, perhaps advertising and reaching the “low-hanging fruit” of human-level AI plus reaping the amazing benefits of such a breakthrough will raise public and political trust in them, as opposed to some “suspicious” corporation or national institute that suddenly builds potential “weapons” of mass destruction.
Well, I’m not at all convinced that substantially self-improving AGI can exist (that is, that will self-improve at such a rate as to quickly gain near complete control of its light cone or something like that). I assign only a small chance to the likelyhood that the first AGI will go foom. Also, if I’ve learned one thing from LW it is that such AI could plausibly be really bad. So I’d rather take a risk averse strategy if it at all possible.
Rule number 1 of voting: it’s done after a thorough debate where every single party has said everything they wanted to say. Not to generalize from fictional evidence, but “Twelve Angry Man” has shown us a pretty good caricature of the dramatic changes that can happen if you prolong the debate just a little bit longer before voting. Which is why educating the public and letting the ideas circulate is so crucial.
You haven’t justified this. What does believeing God guided Evolution have to do with making plans to build a self-improving artificial intelligence?
More importantly, isn’t it better that they know about it, and forbid it or put it under extremely intense scrutiny, rather than they not know about it, and some group developing it and obscurity and botching it?
I think you’re highly delusional about how malleable people’s opinions really are… Are you aware of what’s going on in politics and the religious sphere? As if just talking really thoroughly about AGI and appealing to rationality is going to get the majority of people from all over the world on our side. Are you serious?
The point I made about creationism wasn’t just that most people who believe in god probably won’t want to see one being built, but that you cannot change people’s opinions easily. Even completely ridiculous and unworldly ideas like creationism have hardly budged an inch in the last decade—they are rationalityproof. If you really thoroughly explained to people what this self-improving AGI was good for and how powerful it could really become… they’d totally lose it. They won’t welcome “our robot overlords”, regardless of how nice you make the resulting utopias sound. People fear the unknown and on a gut-level they will immediately reject our idea and rationalize in the blink of an eye why we’re wrong, and crazy, and have to be stopped.
I’m all for thoroughly educating people about rationality (you’ve read my suggestion in the other topic), but seriously getting the majority of people behind us? Sorry, but my psychological model of how people and masses behave tells me that this will never happen. At least not without brain-augmentation, and even then a global 50% +1 vote seems quite unlikely to me.
Would it be better if people knew in detail about self-improving AGI and could objectively discuss this matter in order to rationally make a decision and responsibly vote on whether or not it should be developed? Hell yeah I’d love that! I’d also love to ride on a flying pig but that’s not gonna happen either.
I’d prefer it if you used “mistaken” rather than “delusional”, thank you very much. Ascribing opposing opinions to madness usually signals weakness in your own stance.
Quite, see my next paragraph.
But as a resident of Europe and the Middle East, I see religion and partisanship shriveling into a husk, intelligence and culture extending and growing, triumphant, as they never have before, and the citizenry reclaiming power over the aloof governments, and over their futures.
Humanism wins. And rationalism cannot lose, on the long term, because, as its name indicates, it is the art of being right.
That’s not what I heard at all. Creationism only became acknowledged as a problem recently. Which means it was secure before, and it is now being challenged, and singing it’s swan song. Lack of visible, spectacular budging doesn’t mean that it isn’t crumbling from the inside. And it’s really a problem that is endemic to the USA: across the pond, virtually no one believes in Creationism. I suspect this has something to do with the education of the masses, which is very overlooked in the USA. Once the US society will feel the need to raise their own education level, for any economic reasons, the problems derived from ignorance will just extinguish themselves by sheer lack of combustible.
That’s why the improvement of public, mass education, and the spreading of our Art must be a priority, if not our number one priority. That’s why I said “explaining things thoroughly: by that I mean raising the level of awareness of the general public*.
Here in Spain, France, the UK, the majority of people are Atheists. In the USSR virtually everyone were atheists. Beliefs are extremely malleable. By Raising The Sanity Waterline, we’ll make it so that they are only malleable through empirical evidence, and, if we do it right, people won’t even notice ).
You believe in Transhuman-level cybernetics and brain expansions and you don’t believe we can make pigs fly and carry people on their backs?
Funny, on my second read-through I thought about editing it, but then my mind went “whatever, a healthy ego can probably take it”. No hurt feelings I hope.
I’m also living in the EU, but I’m very aware and constantly following what’s going on in the US, because their erratic development concerns me. As far as evolution and creationism goes, I’m drawing my statistics from the Gallup polls: in the last 30 years creationism went down to 40% from 45%, “evolution through divine intervention” remained stagnant at 40% and “plain natural selection” went up about 5% to 15%.
There is a positive movement here, but I don’t see how in another 30 years the collapse of religion will be imminent. And that’s just the US, to a resident of the Middle East I shouldn’t need to point out, that there are plenty of countries out there much more religious than the US. (Essentially all of them, apart from a few developed countries—and even those countries have usually only around 20% confirmed atheists. Many don’t visit the church, but they’re still holding on to a mountain of superstitious garbage → http://en.wikipedia.org/wiki/Demographics_of_atheism#Europe )
You’re of course also right that here in Europe there’s hardly any creationists—unlike in the US it’s just too damaging to one’s reputation, so people conveniently adapt their views. I doubt however, that this has all that much to do with the quality of education, and a lot more with cultural attitudes towards religion. As far as Europe goes, I can imagine that the church in the biggest countries (France, Germany, Spain, UK...) will be essentially dead in another 30 years, but what has that to say about people’s ability to make rational decisions? There’s a lot more to rationality than not believing in obvious bs.
There’s gonna be close to 9 billion people in 30 years and you think we -a tiny speck of nerds like LW- could hope to reach out and educate a sizable portion (almost half no less) of the world’s people in the art of rationality and in scientific understanding, so we can put AGI up for vote? And it’s not just simply educating them mind you—in a struggle of memes the application of rationality would require them to throw out just about all of their cherished beliefs about life and the world! And you also seem to be forgetting, that the average LW-IQ may perhaps lie somewhere around 120, and that most people aren’t actually all that clever.
I’m an optimist, but I’m afraid without brain-augmentation something like this just isn’t in the realm of possibilities. There’s no way how you could polish our message to a point where it could stick for so many different people.
Damn it! I knew someone would say that and ruin my rhetoric.
As a resident of the Middle East, I can tell you that mentalities are changing fast. Regardless, the attitude towards Science isn’t the same as Christians, since they don’t feel threatened by it, believing that the Qur’an not only isn’t in conflict with science, but actually anticipated some discoveries. They also reclaim the development of modern scientific research as a proud heritage they are enthusiastic to live up to again, and believe researchers should be left alone to investigate, no matter how outrageous the stuff they come up with is (if i remember well think there’s even a command in the Qran specifically to that effect).
As for countries that have been converted to major religions by colonialism, I have a strong feelings that they would actually convert to whatever looks coolest, most Western and most high-status-signalling. We just need to be about 20% cooler than everyone else. Seems manageable.
We should be able to teach rationality to anyone capable of deliberative thought. That is, anyone with an IQ over 70. that the original developers and vanguard be more fast-learning than average is not surprising at all.
Our stuff is simpler, less confusing, far clearer, and far more useful, than anything any religion can teach. I think people could definitely be attracted to our lack of bullshit, if we sell it right.
LOL at the last bit!
I would be interested in knowing where you got your numbers because the statistics I found definitively disagreed with this.
Checks his numbers Forgive me. I should have said the majority of young people (below 30) who, for our uses and purposes, are those who count, and the target demographic. It has come to the point that self-declared Christian kids get bullied and insulted [which is definitely wrong and stupid and not a very good sign that the Sanity Waterline was raised much].
Then again, I have this rule of thumb that I don’t count people who don’t attend church as believers, and automatically lump them into the “atheist in the making” category, a process that is definitely not legitimate nor fair. I sincerely apologize for this, and retract the relevant bits.
Now let’s see. For one thing
The fact that Jedi outnumber Jews in the UK should be a sign that people don’t take that part of the polls very seriously.
That said
This last bit i found particularly troubling because I do not recall metting a single person, in all my time in Spain, who declared themselves a Christian except in name only (as in, embarassingly confessing they only got baptized or went to Communion to please the grandparents). Some entertained some vague fuzziness, but simply telling them a little about “belief in belief” and some reductionist notions has been enough to throw them in serious doubt. I may very well be mistaken, but my perception is that they are really ripe for the taking, and only need to hear the right words.
My perception as a young Arab-European is that the trend is overwhelmingly in the direction of faithlessness, and that it is an accelerating process with no stopping force in sight.