Applause Lights

[Eliezer Yudkowsky]

At the Sin­gu­lar­ity Sum­mit 2007, one of the speak­ers called for demo­cratic, multi­na­tional de­vel­op­ment of ar­tifi­cial in­tel­li­gence. So I stepped up to the micro­phone and asked:

Sup­pose that a group of demo­cratic re­pub­lics form a con­sor­tium to de­velop AI, and there’s a lot of politick­ing dur­ing the pro­cess—some in­ter­est groups have un­usu­ally large in­fluence, oth­ers get shafted—in other words, the re­sult looks just like the prod­ucts of mod­ern democ­ra­cies. Alter­na­tively, sup­pose a group of rebel nerds de­vel­ops an AI in their base­ment, and in­structs the AI to poll ev­ery­one in the world—drop­ping cel­l­phones to any­one who doesn’t have them—and do what­ever the ma­jor­ity says. Which of these do you think is more “demo­cratic,” and would you feel safe with ei­ther?

I wanted to find out whether he be­lieved in the prag­matic ad­e­quacy of the demo­cratic poli­ti­cal pro­cess, or if he be­lieved in the moral right­ness of vot­ing. But the speaker replied:

The first sce­nario sounds like an ed­i­to­rial in Rea­son mag­a­z­ine, and the sec­ond sounds like a Hol­ly­wood movie plot.

Con­fused, I asked:

Then what kind of demo­cratic pro­cess did you have in mind?

The speaker replied:

Some­thing like the Hu­man Genome Pro­ject—that was an in­ter­na­tion­ally spon­sored re­search pro­ject.

I asked:

How would differ­ent in­ter­est groups re­solve their con­flicts in a struc­ture like the Hu­man Genome Pro­ject?

And the speaker said:

I don’t know.

This ex­change puts me in mind of a quote from some dic­ta­tor or other, who was asked if he had any in­ten­tions to move his pet state to­ward democ­racy:

We be­lieve we are already within a demo­cratic sys­tem. Some fac­tors are still miss­ing, like the ex­pres­sion of the peo­ple’s will.

The sub­stance of a democ­racy is the spe­cific mechanism that re­solves policy con­flicts. If all groups had the same preferred poli­cies, there would be no need for democ­racy—we would au­to­mat­i­cally co­op­er­ate. The re­s­olu­tion pro­cess can be a di­rect ma­jor­ity vote, or an elected leg­is­la­ture, or even a voter-sen­si­tive be­hav­ior of an ar­tifi­cial in­tel­li­gence, but it has to be some­thing. What does it mean to call for a “demo­cratic” solu­tion if you don’t have a con­flict-re­s­olu­tion mechanism in mind?

I think it means that you have said the word “democ­racy,” so the au­di­ence is sup­posed to cheer. It’s not so much a propo­si­tional state­ment or be­lief, as the equiv­a­lent of the “Ap­plause” light that tells a stu­dio au­di­ence when to clap.

This case is re­mark­able only in that I mis­took the ap­plause light for a policy sug­ges­tion, with sub­se­quent em­bar­rass­ment for all. Most ap­plause lights are much more blatant, and can be de­tected by a sim­ple re­ver­sal test. For ex­am­ple, sup­pose some­one says:

We need to bal­ance the risks and op­por­tu­ni­ties of AI.

If you re­verse this state­ment, you get:

We shouldn’t bal­ance the risks and op­por­tu­ni­ties of AI.

Since the re­ver­sal sounds ab nor­mal, the un­re­versed state­ment is prob­a­bly nor­mal, im­ply­ing it does not con­vey new in­for­ma­tion.

There are plenty of le­gi­t­i­mate rea­sons for ut­ter­ing a sen­tence that would be un­in­for­ma­tive in iso­la­tion. “We need to bal­ance the risks and op­por­tu­ni­ties of AI” can in­tro­duce a dis­cus­sion topic; it can em­pha­size the im­por­tance of a spe­cific pro­posal for bal­anc­ing; it can crit­i­cize an un­bal­anced pro­posal. Link­ing to a nor­mal as­ser­tion can con­vey new in­for­ma­tion to a bounded ra­tio­nal­ist—the link it­self may not be ob­vi­ous. But if no speci­fics fol­low, the sen­tence is prob­a­bly an ap­plause light.

I am tempted to give a talk some­time that con­sists of noth­ing but ap­plause lights, and see how long it takes for the au­di­ence to start laugh­ing:

I am here to pro­pose to you to­day that we need to bal­ance the risks and op­por­tu­ni­ties of ad­vanced ar­tifi­cial in­tel­li­gence. We should avoid the risks and, in­so­far as it is pos­si­ble, re­al­ize the op­por­tu­ni­ties. We should not need­lessly con­front en­tirely un­nec­es­sary dan­gers. To achieve these goals, we must plan wisely and ra­tio­nally. We should not act in fear and panic, or give in to techno­pho­bia; but nei­ther should we act in blind en­thu­si­asm. We should re­spect the in­ter­ests of all par­ties with a stake in the Sin­gu­lar­ity. We must try to en­sure that the benefits of ad­vanced tech­nolo­gies ac­crue to as many in­di­vi­d­u­als as pos­si­ble, rather than be­ing re­stricted to a few. We must try to avoid, as much as pos­si­ble, vi­o­lent con­flicts us­ing these tech­nolo­gies; and we must pre­vent mas­sive de­struc­tive ca­pa­bil­ity from fal­ling into the hands of in­di­vi­d­u­als. We should think through these is­sues be­fore, not af­ter, it is too late to do any­thing about them . . .

[Ray]

You have, I think, come upon the essence of mod­ern poli­ti­cal speeches.

[theguyfromoverthere]

I was go­ing to say this as well. Your last para­graph here is like ev­ery pres­i­den­tial speech that I’ve ever watched.

[Kevin92]

I voted for Justin Trudeau but DAMN! Listen to his speeches! They’re ter­rible!

[David_J._Balan]

The democ­racy booster prob­a­bly meant that peo­ple with lit­tle poli­ti­cal power should not be ig­nored. And that’s not an empty state­ment; peo­ple with lit­tle poli­ti­cal power are ig­nored all the time.

[Gray]

Ac­tu­ally, that seems to be an ex­tremely empty state­ment. “Hav­ing lit­tle poli­ti­cal power” seems to im­ply, and is im­plied by, “be­ing ig­nored”. I wouldn’t doubt that the two pred­i­cates are co­ex­ten­sive. Since peo­ple with lit­tle poli­ti­cal power are, by defi­ni­tion, ig­nored; say­ing that peo­ple with lit­tle poli­ti­cal power should not be ig­nored makes as much sense as say­ing that squares should be cir­cu­lar.

But maybe I’m not be­ing very char­i­ta­ble here. You can make the shape that was once square more cir­cu­lar, only as long as you note that the shape isn’t a square any­more. Similarly, peo­ple with lit­tle poli­ti­cal power can, over time, gain more poli­ti­cal power, which is a pos­i­tive thing. But even if ev­ery­one has an equal amount of poli­ti­cal power, the propo­si­tion that “peo­ple with lit­tle poli­ti­cal power are ig­nored” would still be true, even if the pred­i­cates con­tain the null set.

[CuSithBell]

I dis­agree.

Even if your in­ter­pre­ta­tion of these terms were ac­cu­rate, “the el­e­ments of this set should (in the fu­ture) not be el­e­ments of this set” isn’t an empty state­ment.

Se­cond, a benev­olent dic­ta­tor (or, say, an FAI) could cer­tainly ad­vance the in­ter­ests of a group with ab­solutely no say in what said dic­ta­tor does.

[Gray]

Eeek, I think the differ­ences in in­ter­pre­ta­tions are due to the de re /​ de dicto dis­tinc­tion.

Com­pare the fol­low­ing trans­la­tions of the state­ment “peo­ple with­out poli­ti­cal power should not be ig­nored.”

De dicto: “It should not be the case that any per­son with­out poli­ti­cal power is also a per­son who is ig­nored.”

De re: “If there is a per­son with­out poli­ti­cal power, then that per­son should not be ig­nored.”

If the two pred­i­cates in the de re in­ter­pre­ta­tion (“per­son with­out poli­ti­cal power” and “per­son who is ig­nored”) are co­ex­ten­sive, and thus equiv­a­lent, we should be able to sub­sti­tute like terms and de­rive “If there is a per­son with­out poli­ti­cal power, then that per­son should not be with­out poli­ti­cal power.” Given that I wanted to use the more char­i­ta­ble in­ter­pre­ta­tion, this is the in­ter­pre­ta­tion I should use, and so you’re cor­rect :)

But look what hap­pens to the de dicto in­ter­pre­ta­tion when you sub­sti­tute like terms. It turns into “It should not be the case that a per­son with­out poli­ti­cal power is a per­son with­out poli­ti­cal power.” This is the sort of thing I was ob­ject­ing to, to be­gin with. But it was the wrong in­ter­pre­ta­tion, and thus my er­ror.

(Yeah, I de­cided to go into an ex­ten­sive anal­y­sis here mainly to re­fine my logic skills and in case any­one else is in­ter­ested. Math­e­mat­i­ci­ans, I sup­pose, would prob­a­bly not have stud­ied the de re /​ de dicto dis­tinc­tion; mainly be­cause I don’t see much rele­vance to math­e­mat­ics.)

[CuSithBell]

Huh! Thanks for the thor­ough anal­y­sis :) I’d say the most likely in­tent be­hind the state­ment is that peo­ple with di­rect poli­ti­cal power should use it for the benefit of those with­out di­rect poli­ti­cal power—i.e. elected offi­cials and so forth should provide sup­port for minor­ity groups with­out much vot­ing power. In which case your ini­tial thought that they in­tended a “de dicto” read­ing could be right!

Did I tip my hand about be­ing a math­e­mat­i­cian by men­tion­ing set the­ory? ;)

[Robin_Hanson2]

Alas, for most au­di­ences I think you would find no one laugh­ing even af­ter an en­tire ap­plause light speech.

[patrissimo]

Yeah, but you’d get lots of ap­plause!

[Chrysophylax]

I tried this for my vale­dic­toral speech and I gave up af­ter about 15 sec­onds due to the laugh­ter.

My preferred method is to use long sen­tences, to speak slowly and se­ri­ously, with great em­pha­sis, and to wave my hands in small cir­cles as I speak. If you don’t speak to this au­di­ence reg­u­larly, it is also a good idea to em­pha­sise how grate­ful you are to be asked to speak on such an im­por­tant oc­ca­sion (and it is a very im­por­tant oc­ca­sion...). You get bonus points for us­ing the phrase “just so chuffed”, es­pe­cially if you use it re­peat­edly (a tech­nique I learned from my old head­mas­ter, who never ex­pressed satis­fac­tion in any other way while giv­ing speeches).

I also recom­mend this tech­nique, this way of speak­ing, to any­one who wishes to wind up, by which I mean an­noy or ir­ri­tate, a fam­ily mem­ber. It’s quite effec­tive when used con­sis­tently, even if you only do it for a minute or two. Don’t you agree?

[Matt_Simpson]

Ev­i­dence: any grad­u­a­tion speech I’ve ever been sub­ject to.

[Peter_de_Blanc]

I re­mem­ber at the AGIRI work­shop in DC last year, Alexei Sam­sonovich talked about sort­ing a list of English words along two di­men­sions—“valence” and “arousal,” in­di­cat­ing some com­po­nent of the emo­tional re­sponse which words evoke.

Maybe au­di­ences re­spond to speeches by sum­ming the emo­tion vec­tors of each word in the speech, rather than pars­ing sen­tences.

Quick test: who here is ex­cited by the prospects of an­thropic quan­tum com­put­ing?

[Arthur1981]

What I find in­ter­est­ing is that there are some ob­vi­ous par­allels be­tween ap­plause lights and Bar­num state­ments—so named af­ter P.T. Bar­num.

Bar­num state­ments are es­sen­tially state­ments which any­one can ap­ply to them­selves as true, which es­sen­tially say noth­ing, and which feel unique to each in­di­vi­d­ual hear­ing them­selves de­scribed that way.

Bar­num state­ments are a stock-in-trade of cold-read­ers such as men­tal­ists and psy­chics. It seems to me that ap­plause lights are noth­ing more than the ab­stract, im­per­sonal ver­sion of the same phe­nom­ena; or per­haps the same phe­nom­ena used in a rhetor­i­cal and ide­olog­i­cal ap­pli­ca­tion.

[shminux]

an­thropic quan­tum computing

I’d think that it came out of a ran­dom ab­stract gen­er­a­tor like snarxiv.

[JohnWittle]

an­thropic quan­tum com­put­ing? if i were flip­ping through the chan­nels and heard that phrase ut­tered by some­one who looked like he was giv­ing a speech, i would be im­me­di­ate in­ter­ested in learn­ing more and would definitely stay on the chan­nel. I have no idea what the phrase means, but my im­me­di­ate guesses are in­deed ex­cit­ing.

[Vladimir_Nesov2]

Such speech could the­o­ret­i­cally perform “bring­ing to at­ten­tion” func­tion. Chunks of “bring­ing to at­ten­tion” are equiv­a­lent to any kind of knowl­edge, it’s just an in­effi­cient form, and ab­nor­mal­ity of that speech in its ut­ter in­effi­ciency, not lack of con­tent. Peo­ple can bear such talk as similar in­effi­ciency can be pre­sent in other talks in differ­ent form. Ineffi­ciency makes it much sim­pler to obfus­cate elud­ing cer­tain top­ics.

[michael_vassar3]

I’m pretty sure that many peo­ple and or­ga­ni­za­tions rou­tinely DO ar­gue that “we shouldn’t bal­ance the risks and op­por­tu­ni­ties of X”. In ethics, de­on­tolog­i­cal sys­tems claim this. In policy, en­vi­ron­men­tal­ists are the first ex­am­ple that spring to mind, though they have been get­ting sub­stan­tially bet­ter in the last few years. Rad­i­cal paci­fists like Gandhi have of­ten been praised for as­sert­ing that peo­ple should not bal­ance the risks and op­por­tu­ni­ties of war. More broadly, dis­play of this at­ti­tude seems to me to be nec­es­sary for any­one who is at­tempt­ing to por­tray that they are ex­traor­di­nar­ily “vir­tu­ous” as virtue is nor­mally un­der­stood, at least in our broadly Chris­tian de­rived civ­i­liza­tion. I ac­tu­ally think that it would be a good idea to try pre­sent­ing all ap­plause lights, but I think that it has been done. “The Gen­tle Art of Ver­bal Self Defense” claims in the ap­pendix that such a speech has been writ­ten and pre­sented to ap­plause on a va­ri­ety of top­ics. It seems to me though that the speech you were propos­ing above was ac­tu­ally an en­dorse­ment of a rea­son­able set of meta-poli­cies which are in fact gen­er­ally not en­gaged in, and was thus sub­stan­tive, not empty, so I’m not sure it counts.

[Anders_Sandberg]

David’s com­ment that we shouldn’t ig­nore peo­ple with lit­tle poli­ti­cal power is a bit prob­le­matic. Peo­ple who are not ig­nored in a poli­ti­cal pro­cess have by defi­ni­tion some poli­ti­cal power; who­ever is ig­nored lacks power. So the mean­ing be­comes “peo­ple who are ig­nored are ig­nored all the time”. The only way to han­dle it is to never ig­nore any­body on any­thing. So please tell me your views on whether Solna munci­pal­ity in Swe­den should spend more money on the stairs above the sta­tion, or a traf­fic light—oth­er­wise the de­ci­sion will not be fully demo­cratic.

I won­der if the sen­si­tivity for ap­plause lights is differ­ent in differ­ent cul­tures. When I lec­tured in Madrid I found mine and sev­eral friend’s speeches fall rel­a­tively flat, de­spite be­ing our nor­mally suc­cess­ful “stan­dard speeches”. But a few oth­ers got roar­ing re­sponses at the ap­plause lights—we were sim­ply not turn­ing them on brighly enough. The re­ward of a roar­ing ap­plause is of course enough to bias a speaker to start pour­ing on more ap­plause lights.

Hmm, was my use of “bias” above just an ap­plause light for Over­com­ing Bias?

[bigjeff5]

The re­ward of a roar­ing ap­plause is of course enough to bias a speaker to start pour­ing on more ap­plause lights. Hmm, was my use of “bias” above just an ap­plause light for Over­com­ing Bias?

Per­haps a bet­ter word would be “train”.

[Luke_G.]

Eliezer’s noth­ing-but-ap­plause-lights speech sounds strangely like ev­ery State of the Union ad­dress I’ve ever heard...

[Arnold_Kling]

See also Trust Cues.

[NickRetallack]

When I click that link, my browser down­loads a file called redi­rect.php.

[Michael_Rooney]

Rather than just “ap­plause lights”, slo­ga­neer­ing of­ten is a cue to group-iden­ti­fi­ca­tion. Cf. post­mod­ern text gen­er­a­tors.

[Cihan_Baran]

This post re­minds me of Ge­orge Or­well’s es­say “Poli­tics and The English Lan­guage”.

[J_Thomas]

“The democ­racy booster prob­a­bly meant that peo­ple with lit­tle poli­ti­cal power should not be ig­nored. And that’s not an empty state­ment; peo­ple with lit­tle poli­ti­cal power are ig­nored all the time.”

But isn’t it pre­cisely the peo­ple with lit­tle poli­ti­cal power who can most safely be ig­nored?

[bigjeff5]

In stan­dard democ­racy, yes, that is the case.

Perfect democ­racy is pure ma­jor­ity rule. Through his­tory we have learned that this is prob­a­bly the worst pos­si­ble idea for a form of gov­ern­ment. The mob has no con­cern for those who are not in the mob, and the ap­a­thy of the crowd can lead to some hor­rific con­se­quences for those in the minor­ity.

This is why most democ­ra­cies are not re­ally democ­ra­cies, but have strong con­straints that boost the power of the weak­est mem­bers to pre­vent them from be­ing over­ruled on ev­ery de­ci­sion, while still giv­ing the ma­jor­ity the larger share of the power.

For ex­am­ple, in the US the demo­cratic pro­cess is split be­tween two houses, The House of Rep­re­sen­ta­tives, which is pop­u­la­tion based and rep­re­sents ma­jor­ity rule, and The Se­nate, for which each state gets only two rep­re­sen­ta­tives re­gard­less of pop­u­la­tion. That bal­ances the power while still giv­ing the ma­jor­ity the ma­jor­ity of the power.

It’s con­straints similar to this (ev­ery­one does it differ­ently, the point is that you always need to do it) that al­low demo­crat­i­cally based sys­tems to work. In the US we also put in a pres­i­dent to make sure things get done, and then went as far out­side the demo­cratic sys­tem as the founders were com­fortable with to in­stall the third con­straint on the sys­tem—the courts.

It could work just fine if there were plenty of well thought out con­straints on it, but “democ­racy” by it­self prob­a­bly would not work at all; it rarely ever does. There­fore, say­ing “democ­racy” with­out any in­ten­tion of dis­cussing it is clearly just an ap­plause word. Either that, or the man was to­tally ig­no­rant. Leave it to some­one like that to re­quire the ab­solute de­struc­tion of a ma­jor effort like AGI just to learn the pit­falls of democ­racy that have been learned over and over and over again.

[omeganaut]

But that in now way im­plies that they should be ig­nored.

[thomblake]

But that in now way im­plies that they should be ig­nored.

It at least to some ex­tent im­plies that they should be ig­nored. To illus­trate:

Some­one who is has great poli­ti­cal power should not be ig­nored. This state­ment is not vac­u­ous; it is in­stead mak­ing a worth­while state­ment of fact. Given that, we know that peo­ple who do not have great poli­ti­cal power should be ig­nored to a greater ex­tent than peo­ple who do have great poli­ti­cal power. Thus, that one does not have great poli­ti­cal power (at least weakly) im­plies that one should be ig­nored (ce­teris paribus). This con­tra­dicts the claim “That in no way im­plies that they should be ig­nored” (em­pha­sis added).

As a side note, the com­ment you’re re­spond­ing to was left in 2007, and even on a differ­ent web­site. As a gen­eral rule, un­less you’re mak­ing a sig­nifi­cant con­tri­bu­tion, it’s not worth re­spond­ing to com­ments that were left be­fore 2009.

If you do be­lieve the par­ent com­ment is a worth­while con­tri­bu­tion, I’d sug­gest cor­rect­ing “now” to “no” (as­sum­ing that’s what you meant).

[James_Bach]

Cu­ri­ously Eliezer, I feel like ap­plaud­ing. Good post.

[Shakespeare's_Fool]

Eliezer,

Thank you for the quo­ta­tion:

“We be­lieve that we are already liv­ing in a democ­racy, al­though some fac­tors are still miss­ing, such as the ex­pres­sion of the peo­ple’s will”

I hope some­one can tell us who said it.

John

[Patri_Friedman]

It might not con­vey in­for­ma­tion, but I bet you could get thun­der­ous ap­plause. Often, the lat­ter out­weighs the former when it comes t the goals of a speech.

[jeff_gray2]

link to 1981 Time mag­a­z­ine in­ter­view with the pres­i­dent of Ar­gentina—source of Eliezer’s quote about democ­racy ab­sent the peo­ple’s will.

http://​​www.time.com/​​time/​​mag­a­z­ine/​​ar­ti­cle/​​0,9171,954853,00.html?pro­moid=googlep

[Leonard]

The sub­stance of a democ­racy is the spe­cific mechanism that re­solves policy con­flicts. … What does it mean to call for a “demo­cratic” solu­tion if you don’t have a con­flict-re­s­olu­tion mechanism in mind?

I think that for many peo­ple the “sub­stance” of democ­racy is not the spe­cific mechanism, but rather the gen­eral mechanism, and the na­ture of the out­put. The mechanism must in­clude at least some for­mal rep­re­sen­ta­tion of ev­ery mem­ber. The de­tails of this don’t mat­ter so much: it might be di­rect vot­ing (strictly equal power), or it might be a rep­re­sen­ta­tive sys­tem (so long as the reps for each voter are more or less equal in power). And the gen­eral na­ture of the out­put is that it should be fair. Ex­actly what fair is, is a good ques­tion, and prob­a­bly varies a lot. But at least this: con­flicts should not always be re­solved in fa­vor of the same per­son or group or class.

This is not a par­tic­u­larly well-defined no­tion; clearly it does not res­onate with you, who want a stric­ter defi­ni­tion. But it is hardly a mean­ingless no­tion, ei­ther. It is not an ap­plause sign.

It is also, I think, a much more use­ful con­cept than you seem to have in mind. You are hung up on speci­fics: “the re­s­olu­tion pro­cess can be a di­rect ma­jor­ity vote, or an elected leg­is­la­ture, or even a voter-sen­si­tive be­hav­ior of an AI, but it has to be some­thing.” Yes, in any ac­tual pro­ject for de­vel­op­ing AI, it would have to be some­thing, and some­thing spe­cific. But speci­fi­cally which of these meth­ods (or an in­finity of other spe­cific im­ple­men­ta­tions of “democ­racy”) did not mat­ter to the speaker you re­fer to.

[pnrjulius]

But what is re­ally that it didn’t MATTER, or sim­ply that he didn’t KNOW?

I think it was the lat­ter—what’s more, it didn’t even oc­cur to him to ask the ques­tion. He seemed to think that say­ing “demo­cratic” was enough.

[Miguel]

I know where your quote came from: http://​​www.time.com/​​time/​​mag­a­z­ine/​​ar­ti­cle/​​0,9171,954853,00.html?promoid

It’s from “Pres­i­dent Roberto Ed­uardo Viola, formerly Ar­gentina’s army com­man­der in chief”.

It’s an an­swer to the first ques­tion in the in­ter­view:

“Q. How soon do you ex­pect Ar­gentina to be re­turned to demo­cratic gov­ern­ment?

A. We be­lieve we are already within a demo­cratic sys­tem. Some fac­tors are still miss­ing, like the ex­pres­sion of the peo­ple’s will, but nev­er­the­less we still think we are within a democ­racy. We say so be­cause we be­lieve these two fun­da­men­tal val­ues of democ­racy, free­dom and jus­tice, are in force in our coun­try. There are, it is true, sev­eral con­di­tion­ing as­pects as re­gards poli­ti­cal or union ac­tivity, but in­di­vi­d­ual free­dom is nowhere in­fringed in an out­stand­ing man­ner.”

BTW, I googled it. Ap­par­ently my Google-fu is bet­ter than yours ;) (But I do ap­plaud your ex­cel­lent mem­ory, or ele I wouldn’t be able to find it).

And keep up with the great posts. I’m a daly reader of this blog.

• Miguel

[Alan_Crowe]

[rhetor­i­cal pose] We shouldn’t bal­ance the risks and op­por­tu­ni­ties of AI. En­thu­si­asts for AI are bi­ased. They un­der es­ti­mate the difficul­ties. They would not be so en­thu­si­as­tic if they grasped how dis­ap­point­ing progress is likely to be. De­trac­tors of AI are also bi­ased. They un­der es­ti­mate the difficul­ties too. You will have a hard time con­vinc­ing them of the difficul­ties, be­cause you would be try­ing to pur­suade them that they had been fright­ened of shad­ows.

So there are few op­por­tu­ni­ties which are likely to be al­to­gether lost if we hang back through un­nec­es­sary fear. [/​rhetor­i­cal]

Well, I hap­pen to be­lieve the two para­graphs above, but dis­tinct from the ques­tion of whether I am right or not is the ques­tion of whether the phrase “We need to bal­ance the risks and op­por­tu­ni­ties of AI.” means some­thing or whether it is merely an ap­plause light.

I think it is triv­ially true that we need to bal­ance the ac­tual risks and ac­tual op­por­tu­ni­ties of AI. There is room for dis­agree­ment about whether we need to bal­ance the per­ceived risks and per­ceived op­por­tu­ni­ties. If per­cep­tions are ac­cu­rate we should, but there is scope to say, for ex­am­ple, that the com­mon per­cep­tion is wrong and a rogue AI will in fact be quite stupid and eas­ily un­plugged. This opens the way to a de­cod­ing of lan­guage in which

o We need to bal­ance the risks and op­por­tu­ni­ties of AI.

is the po­si­tion that we are as­sess­ing the risks and op­por­tu­ni­ties cor­rectly and

o We shouldn’t bal­ance the risks and op­por­tu­ni­ties of AI.

is the po­si­tion that we are as­sess­ing the risks and op­por­tu­ni­ties in­cor­rectly and should fol­low a differ­ent path from that in­di­cated by our in­ac­cu­rate as­sess­ments. Such a po­si­tion needs flesh­ing out with a ri­val ac­count of the risks and op­por­tu­ni­ties.

One ques­tion that I dwell on is “how do in­tel­li­gent and well-in­ten­tion per­sons fall to quar­rel­ling?”. The idea of an Ap­plause Light is illu­mi­nat­ing, but I think it is also quite tan­gled. There is the am­bi­guity be­tween whether a phrase is an Ap­plause Light or a Policy Pro­posal. I sus­pect that the core prob­lem is that it is awfully tempt­ing to ex­ploit this am­bi­guity rhetor­i­cally, de­liber­at­ing cod­ing ones policy pro­pos­als in lan­guage that also func­tions as an Ap­plause Light so that they come across as ob­vi­ously cor­rect.

The fun starts when one does this sub­con­sciously and some-one else thinks it is de­liber­ate and takes offence. Once this hap­pens there is lit­tle chance of dis­cov­er­ing the ac­tual dis­ag­gree­ment (which might be about the ac­cu­racy of risk as­sess­ments) for the con­ver­sa­tion will be de­railed into meta-con­ver­sa­tions about empty phrases and rhetoric.

[bigjeff5]

o We need to bal­ance the risks and op­por­tu­ni­ties of AI.

is the po­si­tion that we are as­sess­ing the risks and op­por­tu­ni­ties cor­rectly and

o We shouldn’t bal­ance the risks and op­por­tu­ni­ties of AI.

is the po­si­tion that we are as­sess­ing the risks and op­por­tu­ni­ties in­cor­rectly and should fol­low a differ­ent path from that in­di­cated by our in­ac­cu­rate as­sess­ments. Such a po­si­tion needs flesh­ing out with a ri­val ac­count of the risks and op­por­tu­ni­ties.

I don’t get that at all. If “We shouldn’t bal­ance the risks and op­por­tu­ni­ties of AI” means they are be­ing as­sessed in­cor­rectly, isn’t that a part of bal­anc­ing the risks and op­por­tu­ni­ties of AI? I don’t see how you can get that out of the state­ment. If they are be­ing done in­cor­rectly, then in the dis­cus­sion of the risks and op­por­tu­ni­ties you say “No, you’re do­ing it wrong, you need to look at it like this blah blah blah”.

When you say “We shouldn’t bal­ance the risks and op­por­tu­ni­ties of AI” it means to stop mak­ing an as­sess­ment al­to­gether. It says noth­ing about con­tin­u­ing to go for­ward with the pro­ject or not. It doesn’t say “Stop the pro­ject! This is all wrong!” That would fall un­der bal­anc­ing the risks and op­por­tu­ni­ties—an as­sess­ment that came against AI.

That’s fool­ish­ness, which is why no one would ever ut­ter the phrase in the first place. That makes the prior phrase an ap­plause phrase, be­cause it is ob­vi­ous to any­one in­volved that such an as­sess­ment is nec­es­sary. You’re only say­ing it be­cause you know peo­ple will nod their head in agree­ment and pos­si­bly clap.

[Polymeron]

It would make sense in the con­text of a strong bias to­ward a spe­cific out­come, e.g. re­li­gious in­dig­na­tion to­ward an idea.

A per­son be­liev­ing that think­ing ma­chines are an abom­i­na­tion would tell you to stop as­sess­ing and for­get the whole idea. A per­son be­liev­ing that AI is the only thing that could pos­si­bly res­cue us from im­mi­nent catas­tro­phe might well tell you to stop an­a­lyz­ing the risks and get on with build­ing the AI be­fore it’s too late.

Either po­si­tion would have a sub­stan­tive po­si­tion that you don’t need to bal­ance the risks and op­por­tu­ni­ties any fur­ther, with­out claiming that you have some er­ror in your as­sess­ment.

[bigjeff5]

Yet build­ing an AI that even­tu­ally de­stroys all mankind, even af­ter it averts this par­tic­u­lar loom­ing catas­tro­phe, could eas­ily be the worse choice. Does the catas­tro­phe we need AI for out­weigh the po­ten­tial dan­gers of a poorly built AI?

It must still be con­sid­ered. You may not have time to con­sider it thor­oughly (as time is now a fac­tor to con­sider), and that must be part of your as­sess­ment, but you still have to weigh the new risks against the po­ten­tial re­ward.

Same with the abom­i­na­tion. Upon what ba­sis is it an abom­i­na­tion? What are the con­se­quences if we cre­ate the abom­i­na­tion? Do we spend a few ex­tra years in pur­ga­tory, or do we burn in hell for all eter­nity?

It still must be con­sid­ered. A few years in pur­ga­tory for a cre­ation that saves mankind from the in­vad­ing squid mon­sters may very much be worth do­ing.

Con­sider the atomic bomb be­fore the first live tests. There were real con­cerns that split­ting the atom could cre­ate an un­stop­pable chain of events which would set the very air on fire, de­stroy­ing the whole world in that sin­gle mo­ment. I can’t re­ally imag­ine a sce­nario that is more dire, and more strongly ar­gues for the ceas­ing of all ar­gu­ment.

Yet they did the math any­way, con­sid­ered the risks (tiny chance of blow­ing up the world) vs the re­ward (end­ing the war that is guaran­teed to kill mil­lions more peo­ple), and de­cided it was worth it to con­tinue.

I still see no ra­tio­nal case for ever halt­ing ar­gu­ment, ex­cept in the case of time for as­sess­ment sim­ply run­ning out (if you don’t act be­fore X, the world blows up—ob­vi­ously you must finish your as­sess­ment be­fore X or it was all pointless). You may weigh the risks vs the op­por­tu­ni­ties and de­cide the risks are too great, and de­cide not to con­tinue. How­ever, you can not ra­tio­nally cease all ar­gu­ment with­out con­sid­er­a­tion be­cause of a par­tic­u­larly strong or dire ar­gu­ment. To do so is ir­ra­tional.

[Polymeron]

Of course you can cease ar­gu­ment with­out con­sid­er­a­tion—if you deem the risks of con­tin­u­ing con­sid­er­a­tion to out­weigh the benefits of weigh­ing them. For in­stance, if you have 1 minute to try some­thing that would save your life, and you re­quire at least 5 min­utes to prop­erly as­sess any­thing fur­ther, you gen­er­ally can’t af­ford to weigh whether the idea would re­sult in a worse situ­a­tion some­how—be­yond what­ever as­sess­ment you have already made. At that point, the time for as­sess­ment is over.

For the most part, how­ever, I agree with your point. I did not ar­gue that one can ra­tio­nally dis­agree with the state­ment “We need to bal­ance the risks and op­por­tu­ni­ties of AI”; just that they can sincerely say it, and even ar­gue for it. This was a re­sponse to you say­ing that “no one would ever ut­ter the phrase in the first place”. This just strikes me as false.

Never un­der­es­ti­mate the power of hu­man stu­pidity ;)

[bigjeff5]

You’re right, in that re­gard I was cer­tainly mis­taken.

[Viktor Riabtsev]

Upvoted for the “oops” mo­ment.

[Venkat]

That was kinda hilar­i­ous. I like your re­ver­sal test to de­tect con­tent-free tau­tolo­gies. Since I am work­ing right now on a piece of AI-poli­ti­cal-fic­tion (in­volv­ing vot­ing rights for ar­tifi­cial agents and ques­tions that raises), I was thrown for a mo­ment, but then tuned in to what YOU were talk­ing about.

The ‘Yes, Minister’ and ‘Yes, Prime Minister’ se­ries is full of ex­tended pieces of such con­tent-free di­a­log.

More se­ri­ously though, this is a bit of a straw­man at­tack on the word ‘democ­racy’ be­ing used as dec­o­ra­tion/​group dy­nam­ics cue­ing. You kinda blind-sided this guy, and I sus­pect he’d have a bet­ter an­swer if he had time to think. There is SOME con­tent even to such a woolly-headed sen­ti­ment. Any large group (in­clud­ing large re­search teams) has con­flict, and there is a spec­trum of con­flict re­s­olu­tion rang­ing from dic­ta­to­rial im­po­si­tion to democ­racy through to con­sen­sus.

Whether or not the for­mal scaf­fold­ing is pre­sent, an ac­tivity as com­plex as re­search CANNOT work un­less the con­flict re­s­olu­tion mechanisms are closer to the democ­racy/​con­sen­sus end of the spec­trum. Dic­ta­tors can whip peo­ple’s mus­cles into obe­di­ence, maybe even their lower-end skills (“do this ar­ith­metic or DIE!”), but when you want to en­gage the cre­ativity of a gang of PhDs, it is not go­ing to work un­til there is a mechanism for their dis­sent to be heard and ad­dressed. This means mak­ing the group it­self rep­re­sen­ta­tive (the ‘multi­na­tional’ part) au­to­mat­i­cally brings in the spirit if not the form of demo­cratic dis­courses. So yes, if there are au­to­cen­tric cul­tural bi­ases to­day’s AI re­searchers bring to the game, mak­ing the fund­ing and ex­e­cu­tion multi­na­tional would help. Hav­ing worked on AI re­search as an in­tern in In­dia 12 years ago, and work­ing to­day in re­lated fields here in the US, I can’t say I see any such bi­ases in this par­tic­u­lar field, but per­haps in other fields, mak­ing up multi­na­tional, in­ter­na­tion­ally-funded re­search teams would ac­tu­ally help.

On the flip side, you can have all the mechanisms and still al­low dic­ta­to­rial in­tent to pre­vail. My mod­est take on ru­in­ing demo­cratic meet­ings run on Robert’s Rules:

The 15 laws of Meet­ing Power

[Jamais_Cascio]

C’mon, Eliezar, be fair: iden­tify who the speaker was that you “probed” in this way, so that peo­ple can find the record­ings of the talk and ex­change at sin­ginst.org to de­cide for them­selves how it went.

As you have it above, aside from the para­phras­ing, you omit a cou­ple of im­por­tant parts of my replies. With re­gards to the Rea­son/​Hol­ly­wood com­par­i­son, I go on to say:

“That is, they’re both car­i­ca­tures, and nei­ther one is ter­ribly plau­si­ble or com­plete. There would be some crit­i­cal benefits to the messy pro­cess of the first sce­nario, and some im­por­tant draw­backs to the sec­ond.”

With re­gards to the “I don’t know,” I then say:

“This is a point I’ve tried to make a cou­ple of times here: this is not a solved prob­lem, but it’s an im­por­tant prob­lem, and we need to figure out how to ad­dress it.”

I cer­tainly did not talk about democ­racy with any in­tent of it serv­ing as “ap­plause lights” for my talk—in fact, given the au­di­ence, I ex­pected a semi-hos­tile re­sponse, given my ar­gu­ment against the kind of “rebel nerd” hero­ism self-image a lot of the AGI com­mu­nity seems to have.

[Eliezer Yudkowsky]

BTW, if any­one wants to go to sin­ginst.org and down­load the au­dio, you’ll note that the ac­tual event did not oc­cur the ex­act way I re­mem­bered it, which should sur­prise no one here who knows any­thing about hu­man mem­ory. In par­tic­u­lar, Cas­cio spon­ta­neously pro­vided the Genome Pro­ject ex­am­ple, rather than need­ing to be asked for it.

Gen­er­ally, the rea­son I avoid iden­ti­fy­ing the char­ac­ters in my ex­am­ples is that it feels to me like I’m dump­ing all the sins of hu­mankind upon their un­de­serv­ing heads—I’m pre­sent­ing one er­ror, out of con­text, as ex­em­plar for all the er­rors of this kind that have ever been com­mit­ted, and show­ing none of the good qual­ities of the speaker—it would be like car­i­ca­tur­ing them, if I called them by name.

That said, the rea­son why I picked this ex­am­ple is that, in fact, I was think­ing of Or­well’s “Poli­tics and the English Lan­guage” while writ­ing this post. And as Or­well said:

In the case of a word like democ­racy, not only is there no agreed defi­ni­tion, but the at­tempt to make one is re­sisted from all sides. It is al­most uni­ver­sally felt that when we call a coun­try demo­cratic we are prais­ing it: con­se­quently the defen­ders of ev­ery kind of regime claim that it is a democ­racy, and fear that they might have to stop us­ing that word if it were tied down to any one mean­ing.

If you sim­ply is­sue a call for “democ­racy”, why, no one can dis­agree with that—it would be like dis­agree­ing with a call for ap­ple pie. As soon as you pro­pose a spe­cific mechanism of democ­racy, whether it is Congress pass­ing a law, or an AI pol­ling peo­ple by phone, or gov­ern­ment fund­ing of a large re­search pro­ject whose fi­nal au­thor­ity be­longs to an ap­pointed com­mit­tee of em­i­nent sci­en­tists, et cetera, peo­ple can dis­agree with that, be­cause they can ac­tu­ally vi­su­al­ize the prob­a­ble con­se­quences.

So there is a tremen­dous mo­tive to avoid crit­i­cism, to keep to the safely vague ar­eas where peo­ple will ap­plaud you, and not to make the con­crete pro­pos­als where peo­ple might—gasp! - dis­agree.

Now I do not ac­cuse you too much of this, be­cause you did say “Genome Pro­ject” when challenged in­stead of squirt­ing out an im­mense cloud of ink. But it is why I challenged you to define “democ­racy”. I think that the real value in these dis­cus­sions comes from peo­ple will­ing to make con­crete pro­pos­als and ex­pose them­selves to crit­i­cism.

[capybaralet]

Really bad ex­am­ple...

My im­pres­sion is that democ­racy is see­ing a sharp uptick in at­tacks from elites and in­tel­lec­tu­als. There are many who now be­lieve, e.g., that the US should be more like China (see: the suc­cess of Trump).

As the speaker noted, he ex­pected his speech to be con­tro­ver­sial in that crowd, and in a way, it was, as ev­i­denced by this blog post :)

[Fyrius]

Hum.

When I hear the sort of thing you would call “ap­plause lights”, I don’t always think of that as an ob­vi­ous fact that ev­ery­one in their right mind would agree on. Rather, I get the im­pres­sion the speaker is im­ply­ing that some­one they strongly dis­agree with does be­lieve this ob­vi­ous fact is not true, or that this ridicu­lous no­tion is.

If for ex­am­ple I hear some­one say “we shouldn’t be hug­ging crim­i­nals, we should be lock­ing them up”, I in­ter­pret that as a very one-sided op­po­si­tion to a grossly mis­rep­re­sented op­po­nent who goes a bit eas­ier on con­victs. Of course this per­son wouldn’t liter­ally be­lieve the re­verse that “we should be hug­ging crim­i­nals in­stead of lock­ing them up”, but she might be­lieve some­thing that a bi­got could para­phrase as such with a straight face.

I think this is also the rea­son why the speaker’s sup­port­ers ap­plaud to state­ments like that—it im­plies the is­sue is very sim­ple and clear-cut, only one side (ours) is re­motely sen­si­ble, and you’d have to be in­sane to dis­agree. One-sid­ed­ness feels good. Very blatant one-sid­ed­ness feels even bet­ter.

(Ex­cuse me if this has been said already.)

[NancyLebovitz]

I haven’t seen it laid out so clearly any­where.

The only thing I’d add is that it’s very easy to fall into that er­ror re­flex­ively. It isn’t gen­er­ally a mat­ter of con­scious strat­egy.

[pnrjulius]

Hence, an ap­plause light is a form of straw­man ar­gu­men­ta­tion?

That sounds about right ac­tu­ally.

[Fyrius]

It can be used for that, at least.

[DSimon]

(Hi ev­ery­one; this is my first time post­ing here.)

If some­one de­liv­ered that 100%-ap­plause-light para­graph to me in a speech, my first im­pulse would be to in­ter­pret it as an hon­est at­tempt to re­mind the au­di­ence of ob­vi­ous but not nec­es­sar­ily cur­rently-in-con­text ideas. For ex­am­ple, this state­ment from the mid­dle:

“To achieve these goals, we must plan wisely and ra­tio­nally. We should not act in fear and panic, or give in to techno­pho­bia; but nei­ther should we act in blind en­thu­si­asm.”

Taken liter­ally as a set of as­ser­tions, this re­ally is quite empty of novel or un­ex­pected con­tent. How­ever, di­rected at an au­di­ence of hu­mans, aware of but still vuln­er­a­ble to cog­ni­tive bias, the state­ment above im­plies an­other state­ment which is more use­ful: “We should be care­ful to not act like who, de­spite in­tend­ing not to, pan­icked rather than think­ing pro­duc­tively. We should also be care­ful to not act like whose en­thu­si­asm over­whelmed their nec­es­sary sense of cau­tion, even though they knew the value of that cau­tion.”

Peo­ple who agree with the part of the 1st virtue that says “A burn­ing itch to know is higher than a solemn vow to pur­sue truth” may still some­times need to be re­minded to check them­selves and make sure they’re do­ing the former rather than the lat­ter.

[Document]

This sounds similar to the idea of a “moth­er­hood state­ment” as defined here.

[pnrjulius]

That sec­ond defi­ni­tion ap­plies to most de­pic­tions of tran­shu­man­ism in fic­tion. It’s the rare au­thor who is bold enough to say, “The im­plants that we put in our brains? Yeah, they ac­tu­ally make us bet­ter.”

[TheOtherDave]

Pretty much all the fic­tion I read in which brain im­plants are men­tioned at all treat them as im­prove­ments.

[pnrjulius]

Really? Got any ex­am­ples?

I’ve read some in which the tran­shu­man tech­nolo­gies were am­bigu­ous (had up­sides and down­sides), but I can’t think of any where it was just bet­ter, the way that ac­tual tech­nolo­gies of­ten are—would any of us will­ingly go back to the days be­fore elec­tric­ity and run­ning wa­ter?

[Swimmer963]

I’ve read some in which the tran­shu­man tech­nolo­gies were am­bigu­ous (had up­sides and down­sides), but I can’t think of any where it was just bet­ter, the way that ac­tual tech­nolo­gies of­ten are—would any of us will­ingly go back to the days be­fore elec­tric­ity and run­ning wa­ter?

Hav­ing up­sides and down­sides isn’t the same thing as be­ing am­bigu­ous. Run­ning wa­ter and elec­tric­ity do have down­sides–namely, de­ple­tion of wa­ter ta­bles due to overuse, and pol­lu­tion, re­source de­ple­tion, and pos­si­bly global warm­ing due in part to the efforts re­quired to make elec­tric­ity...But I wouldn’t say that ei­ther tech­nol­ogy is am­bigu­ous. The ad­van­tages pretty clearly out­weigh the dis­ad­van­tages, which are avoid­able with some thought and cre­ativity.

[Nornagest]

would any of us will­ingly go back to the days be­fore elec­tric­ity and run­ning wa­ter?

Well, they’re hardly com­mon, but an­ar­cho-prim­i­tivists do ex­ist.

[TheOtherDave]

Most of Peter Hamil­ton’s stuff comes to mind, for ex­am­ple. Im­plants are just an­other tech­nol­ogy, treated no differ­ently than guns or cars. The Greg Man­del books have a few char­ac­ters who do end up with im­plants that they would pre­fer not to have, but they’re the ex­cep­tions.

[Hul-Gil]

but I can’t think of any where it was just bet­ter, the way that ac­tual tech­nolo­gies of­ten are

I find that a lit­tle ir­ri­tat­ing—for peo­ple sup­pos­edly open to new ideas, sci­ence fic­tion au­thors sure seem fear­ful and/​or dis­ap­prov­ing of fu­ture tech­nol­ogy.

[Nornagest]

Part of me thinks that that’s en­coded into the metaphor­i­cal DNA of the SF genre (or one branch of it) at a very ba­sic level. It’s been con­ven­tional for a while to think of SF as En­light­en­ment and the rest of spec-fic as Ro­man­tic, but the his­tory of the genre’s ac­tu­ally more com­pli­cated than that; Mary Shel­ley, for ex­am­ple, definitely fell on the Ro­man­tic side of the fence, and later writ­ers haven’t ex­actly been shy about fol­low­ing her lead. The tread­ing-in-God’s-do­main mo­tif is a pow­er­ful one, and it’s the bedrock that an awful lot of SF is built on.

[taelor]

Obli­ga­tory Link

[bigjeff5]

Oy, now that you’ve said it, I hear speeches like that at the end all the time. Whole dis­cus­sion be­tween op­pos­ing sides even. Per­haps that’s why I haven’t been able to stand ca­ble news for a while now?

[Polymeron]

When I first read this, I imag­ined a fa­vorite poli­ti­cian (I won’t men­tion who) giv­ing this mock speech.

To my em­bar­rass­ment, I found my­self nod­ding in com­pletely gen­uine en­thu­si­asm. This guy clearly knows what he’s talk­ing about!

(This in turn made me con­sider just how much of this poli­ti­cian’s speeches was similarly com­posed. I came to the con­clu­sion that quite a sig­nifi­cant amount of it was)

...No­body ever told me cog­ni­tive bias would be this an­noy­ing!

[TheOtherDave]

Upvoted be­cause I en­dorse the will­ing­ness to no­tice one’s own bi­ases.

So, next ques­tion, if you’re will­ing: what are three things you could do to re­duce the de­gree to which this sort of empty rhetoric leads you to en­dorse the speaker?

[Polymeron]

TheOtherDave, that is a very con­struc­tive ap­proach :)

I am already prone to re­quiring policy speci­fics from poli­ti­ci­ans and be­ing dis­satis­fied with vague points. But one thing I (and many oth­ers) do have is a ten­dency to note, when hear­ing a few speci­fics in a sea of “gen­eral di­rec­tion” ap­plause cues, is that my own prefer­ence for solu­tions is com­pat­i­ble with the speech; and from com­pat­i­bil­ity, I get hope that they would im­ple­ment it—de­spite a lack of ev­i­dence that they’re even aware of such a solu­tion, much less want to im­ple­ment it. So this is some­thing to be cau­tious of and to note mid-speech.

I could go fur­ther and try to strike from men­tal record any­thing that isn’t speci­fics, mak­ing a point-by-point list of sub­stan­tive state­ments. An easy way to do this is ask “is any­one re­ally con­sid­er­ing do­ing oth­er­wise? No? Then it doesn’t count. Yes? Then why are they?” This method might not always be wise—mo­ti­va­tions and be­liefs are also im­por­tant in try­ing to pre­dict a poli­ti­cian’s fu­ture choices they did not yet ad­dress, and the speech can pro­nounce those. How­ever it would be a good men­tal ex­er­cise when try­ing to eval­u­ate po­si­tions on a spe­cific policy ques­tion.

Lastly, try to sep­a­rate emo­tional jar­gon from ac­tual policy. If your poli­ti­cian says we “need to be pre­pared for the 21st cen­tury”, rec­og­nize the fuzzy ex­cite­ment that this state­ment gives you and squash it—it’s caused by the phrase “21st cen­tury” be­ing linked in your mind with progress and tech­nol­ogy. Wait un­til that poli­ti­cian says they’re go­ing to speci­fi­cally in­vest in tech­nolog­i­cal liter­acy of 8th graders be­fore you give it any sig­nifi­cance, and treat it as sus­pect un­til then. (This is very similar to the first thing I sug­gested, ex­cept it fo­cuses on rec­og­niz­ing an im­me­di­ately trig­gered emo­tion in re­sponse to a phrase, rather than your own mind build­ing sce­nar­ios which then in turn ex­cite you).

I’ll try to re­mem­ber all that for the next speech I hear :P

[TheOtherDave]

I definitely en­dorse track­ing spe­cific pro­pos­als/​sub­stan­tive as­ser­tions, and ex­plic­itly la­bel­ing vague or empty as­ser­tions that nev­er­the­less elicit pos­i­tive feel­ings or in­vite you to pro­ject your own prefer­ences onto the speaker.

I definitely en­dorse ask­ing the “is any­one re­ally con­sid­er­ing do­ing oth­er­wise, and why?” ques­tion.

Some­thing I also find use­ful is ex­plic­itly la­bel­ing im­plied af­fili­a­tions.

E.g., con­sider the differ­ence be­tween “we need to pre­pare our chil­dren with the tools they need to be lead­ers in the 21st cen­tury,” ver­sus “we need to in­still our chil­dren with the val­ues they need to make the right choices in the 21st cen­tury.” They are both empty state­ments—I mean, who would ever claim oth­er­wise? -- but in the U.S. to­day the former sig­nals af­fili­a­tion with teach­ers and thereby im­plies sup­port for pub­lic schools, ed­u­ca­tion fund­ing, etc., while the lat­ter sig­nals some­thing I un­der­stand less clearly.

And those in turn sig­nal al­li­ances with ma­jor poli­ti­cal par­ties, be­cause it’s un­der­stood by most U.S. vot­ers that party A is more closely tied to ed­u­ca­tion and party B to val­ues.

In fact, even if the state­ment in­cludes a spe­cific pro­posal, it is of­ten worth la­bel­ing the im­plied af­fili­a­tion.

[pnrjulius]

It’s in­ter­est­ing; with the con­no­ta­tions and as­so­ci­a­tions in our dis­course, I can ac­tu­ally make some pre­dic­tions about planned poli­cies from those two sup­pos­edly “empty” state­ments.

The former is prob­a­bly go­ing to spend more money on math and sci­ence ed­u­ca­tion.

The lat­ter is prob­a­bly go­ing to fund “faith-based ini­ti­a­tives” or some­thing similarly silly and re­li­gious (but I re­peat my­self), be­cause “val­ues” in Amer­i­can poli­tics is al­most always code for “con­ser­va­tive Evan­gel­i­cal Chris­ti­an­ity”.

So does this mean that they re­ally aren’t empty at all?

[TheOtherDave]

Well, yes, I chose those state­ments pre­cisely be­cause of their con­no­ta­tive af­fili­a­tions.

As for whether they’re re­ally empty… (shrug).

In or­di­nary con­ver­sa­tion I would con­sider “I like lik­able things!” an empty state­ment, but of course it con­veys an enor­mous amount of in­for­ma­tion: that I am ca­pa­ble of con­struct­ing a gram­mat­i­cal English sen­tence, for ex­am­ple, which the Vast ma­jor­ity of equiv­a­lent-mass ag­gre­ga­tions of par­ti­cles in the uni­verse are not. I can use a differ­ent term to de­scribe that cat­e­gory of state­ment if this one is too am­bigu­ous.

[potato]

“Ap­plause Light” is a won­der­ful name for that tac­tic; it’s funny, catchy, and makes the prob­lem with that tac­tic in­tu­itively ob­vi­ous. That term should be fur­ther pro­lifer­ated through­out the in­ter­net if it hasn’t already been. Ad­ding that meme to the av­er­age in­ter­net goer’s reper­toire could have won­der­ful side-effects on the sup­port de­ci­sions of peo­ple in meat space ev­ery­where.

[MarkusRamikin]

That ap­plause-light speech at the end just needs some vari­a­tion, and I’m pretty sure it would fly. I’d re­place about half of the “we should” with some­thing else, like “it is im­por­tant that we”, and “it would be dan­ger­ous to ne­glect” and so on, be­cause right now it’s so repet­i­tive that surely a lot of peo­ple would no­tice and re­al­ize what’s be­ing done.

Or maybe i’m yet again over­es­ti­mat­ing my fel­low hu­man be­ings, as past ex­pe­rience says I am prone to do...

[Jeremygbyrne]

I am tempted to give a talk some­time that con­sists of noth­ing but ap­plause lights

http://​​www.youtube.com/​​watch?v=pxMqSdgB-uA (ap­pro­pri­ately ti­tled “Un­think”).

[jeromeapura]

You got him on nice So­cratic ques­tion. Well, a good ques­tion seeks good idea and elimi­nate inane idea. Nice

[Matvey_Ezhov]

There might be one an­other case: in ca­sual con­ver­sa­tion, some­thing that looks like an ap­plause light, could be just ex­pres­sion of re­cent in­sights of a par­tic­u­lar per­son on the sub­ject. Like, he just yes­ter­day de­duced (based on some frag­ments of ra­tio­nal texts on the web) that we should bal­ance risks and op­por­tu­ni­ties of this. Or, maybe the au­di­ence level on the sub­ject is so low that even the ap­plause-like state­ments do con­vey some in­for­ma­tion.

[buybuydandavis]

“Let’s do ev­ery­thing right.”

Yep. Stan­dard poli­ti­cal speech.

[Maha]

I don’t think these state­ments are en­tirely vac­u­ous. Even when their con­tent is lit­tle more than a tau­tol­ogy, their ac­tual mean­ing is some­thing else en­tirely, at least in poli­tics; they rep­re­sent that the speaker is aware of the jar­gon, will­ing to use it, es­sen­tially mod­er­ate/​”prag­matic” and prone to main­tain­ing the sta­tus quo.

[macronencer]

I couldn’t re­sist adding an­other link as an ex­am­ple of a speech that seems to con­sist al­most en­tirely of ap­plause lights. This one is vin­tage Peter Sel­lers.

http://​​www.youtube.com/​​watch?v=GxBtGuu9BVE

[taelor]

Ap­plause Lights also have more sinister, dark artsy ap­pli­ca­tion: they can be used to bait peo­ple into agree­ing with seem­ingly triv­ial propo­si­tions, which nev­er­the­less cause the tar­get to mod­ify their self image, ren­der­ing them more likely to agree with less triv­ial propo­si­tions in the fu­ture. For ex­am­ple, Cial­dini’s In­fluence re­ports on a study that found that house­holds that had been vis­ited by a vol­un­teer col­lect­ing sig­na­tures in fa­vor of the vague state­ment “keep Cal­ifor­nia beau­tiful” (with­out ever spec­i­fy­ing how this was to be ac­com­plished) were much more likely to agree to promi­nently dis­play a large, ugly sign read­ing “pre­vent drunk driv­ing” on their yards than house­holds that hadn’t been so vis­ited.

[olalonde]

can con­vey new in­for­ma­tion to a bounded rationalist

Why limit it to bounded ra­tio­nal­ists?

[PhilGoetz]

How would differ­ent in­ter­est groups re­solve their con­flicts in a struc­ture like the Hu­man Genome Pro­ject?

Oh, my, the un­in­ten­tional hu­mor of that speaker’s com­ment. There’s an en­tire book writ­ten on how groups re­solved their con­flicts in the Hu­man Genome Pro­ject, The Genome War: They didn’t. The out­come was a hor­rific case study of how sci­ence re­ally “works” to­day.

[poi99]

You’ll of­ten see fem­i­nists on the In­ter­net point­ing out how just about ev­ery­one seems to be in fa­vor of “gen­der equal­ity” and yet hardly any­one of ei­ther gen­der self-iden­ti­fies as fem­i­nist any­more, even though gen­der equal­ity is what fem­i­nism claims to be all about.

This ar­ti­cle ex­plains that dis­con­nect. “Equal­ity” is an ap­plause light. It’s some­thing we can all agree is great in the ab­stract, but as soon as some­one starts talk­ing speci­fics, the ap­plause thins and we all go back to be­ing po­larized again be­cause ev­ery­one’s idea of what equal­ity, es­pe­cially so­cial and eco­nomic equal­ity, ac­tu­ally en­tails is differ­ent.

[Nisan]

“I am here to pro­pose to you to­day that we need to bal­ance the risks and op­por­tu­ni­ties of ad­vanced Ar­tifi­cial In­tel­li­gence...”

Seven years later, this open let­ter was signed by lead­ers of the field. It’s amus­ing how similar it is to the above speech, es­pe­cially con­sid­er­ing how it ac­tu­ally marked a ma­jor mile­stone in the ad­vance­ment of the field of AI safety.

[Jiro]

Then what kind of demo­cratic pro­cess did you have in mind?

The speaker replied:

Some­thing like the Hu­man Genome Pro­ject—that was an in­ter­na­tion­ally spon­sored re­search pro­ject.

I asked:

How would differ­ent in­ter­est groups re­solve their con­flicts in a struc­ture like the Hu­man Genome Pro­ject?

And the speaker said:

I don’t know.

In this old post, Eliezer is be­ing in­suffi­ciently char­i­ta­ble and not steel­man­ning.

It is pos­si­ble that I know that X can do A, but I don’t know how X can do A. “Look at X and do A similarly to that” may be a rea­son­able re­sponse when I am asked how to do A (or when I am told that A is just im­pos­si­ble). It may fail in some cases (such as when I am in­cor­rect in my as­sump­tion that X and the cur­rent situ­a­tion are similar enough), but it isn’t de­void of con­tent to say that, and is more than an ap­plause light.

[zby]

Sounds like https://​​en.wikipe­dia.org/​​wiki/​​Float­ing_signifier

[Robin Caesar]

Hi, i made this ger­man trans­la­tion if any­one is in­ter­ested:

Ap­plaus-Lichter

Pseudo-Überzeugungen

Von Eliezer_Yudkowsky

11. Septem­ber 2007

Auf dem Sin­gu­lar­ity Sum­mit 2007 rief einer der Red­ner zu einer demokratischen, multi­na­tionalen En­twick­lung der KI auf. Also trat ich ans Mikrofon und fragte:

Angenom­men, eine Gruppe demokratischer Repub­liken bildet ein Kon­sor­tium zur En­twick­lung der KI, und während des Prozesses gibt es viel poli­tisches Tak­tieren—einige In­ter­es­sen­grup­pen haben einen ungewöhn­lich großen Ein­fluss, an­dere wer­den über den Tisch gezo­gen—mit an­deren Worten, das Ergeb­nis sieht aus wie die Pro­dukte mod­erner Demokra­tien. Alter­na­tives Szenario: Angenom­men, eine Gruppe von Re­bel­len-Nerds en­twick­elt eine KI in ihrem Kel­ler und weist der KI an, mit je­dem auf der Welt eine Um­frage zu machen (dafür bekommt jeder der noch keines hat ein Mo­biltele­fon) - und dann das zu machen, was die Mehrheit wün­scht. Welche von diesen Op­tio­nen ist Ihrer Mei­nung nach “demokratischer”, und wür­den Sie sich bei bei­den sicher füh­len?

Ich wol­lte her­aus­fin­den, ob er an die prag­ma­tische Angemessen­heit des demokratischen poli­tischen Prozesses glaubte oder ob er an die moral­ische Recht­mäßigkeit der Stimmab­gabe glaubte. Der Sprecher antwortete:

Das er­ste Szenario klingt wie ein Leitar­tikel im Rea­son-Magazin und das zweite klingt wie ein Hol­ly­wood-Plot.

Ver­wirrt fragte ich:

An was für einen demokratischen Prozess haben Sie gedacht?

Der Sprecher antwortete:

So et­was wie das Hu­man Genome Pro­ject—das war ein in­ter­na­tional gespon­sertes Forschungspro­jekt.

Ich habe gefragt:

Wie wür­den ver­schie­dene In­ter­es­sen­grup­pen ihre Kon­flikte in einer Struk­tur wie dem Hu­man Genome Pro­ject lösen?

Und der Sprecher sagte:

Ich weiß es nicht.

Dieser Aus­tausch er­in­nert mich an ein Zi­tat (das ich bei Google nicht ge­fun­den habe, aber das von Jeff Gray und Miguel ge­fun­den wurde) von ir­gen­deinem Dik­ta­tor, der gefragt wurde, ob er be­ab­sichtige, seinen Bana­nen-Repub­lik in Rich­tung Demokratie zu be­we­gen:

Wir glauben, dass wir uns bere­its in einem demokratischen Sys­tem befin­den. Einige Fak­toren fehlen noch, wie die freie Mei­nungsäußerung.

Der Kern einer Demokratie ist der spez­ifische Mechanis­mus, der poli­tische Kon­flikte löst. Wenn alle Grup­pen die gle­iche bevorzugte Poli­tik hät­ten, wäre keine Demokratie erforder­lich—wir wür­den au­toma­tisch zusam­me­nar­beiten. Der Kon­flik­tlö­sungsprozess kann eine di­rekte Stim­men­mehrheit oder eine gewäh­lte Leg­is­la­tive oder sogar ein wäh­lersen­si­tives Ver­halten einer KI sein, aber es muss je­doch et­was sein. Was be­deutet es, eine “demokratische” Lö­sung zu fordern, wenn Sie keinen Kon­flik­tlö­sungsmechanis­mus im Sinn haben?

Ich denke, es be­deutet, dass das Wort “Demokratie” gesagt wurde, und dass das Publikum Beifall klatschen soll. Es ist nicht so sehr eine aus­sagekräftige Aus­sage, als ein Äquiv­a­lent des “Ap­plause” -Lichts, das einem Stu­diop­ub­likum sagt, wann es klatschen soll.

Dieser Fall ist nur in­sofern be­merkenswert, als dass ich das Ap­plaus-Licht mit einem poli­tischen Vorschlag ver­wech­selt habe, mit an­schließen­der al­lge­meiner Pein­lichkeit. Die meisten Ap­plaus-Lichter sind offen­sichtlicher und kön­nen durch einen ein­fachen Umkehrtest erkannt wer­den. Angenom­men, je­mand sagt:

Wir müssen die Risiken und Chan­cen der KI aus­bal­ancieren.

Wenn Sie diese Aus­sage umkehren, er­halten Sie:

Wir sol­lten die Risiken und Chan­cen der KI nicht aus­bal­ancieren.

Da die Umkehrung ab­nor­mal klingt, ist die nicht umgekehrte Aus­sage wahrschein­lich nor­mal, was be­deutet, dass keine neuen In­for­ma­tio­nen über­mit­telt wer­den. Es gibt viele le­gi­t­ime Gründe, um einen Satz auszus­prechen, der isoliert nicht aus­sagekräftig wäre. “Wir müssen die Risiken und Chan­cen der KI aus­bal­ancieren” kann ein Diskus­sion­s­thema ein­führen; es kann die Bedeu­tung eines spez­ifischen Vorschlags für die Ab­wä­gung be­to­nen; es kann einen unaus­ge­wo­ge­nen Vorschlag kri­tisieren. Das Verknüpfen mit einer nor­malen Be­haup­tung kann einem eingeschränk­ten Ra­tion­al­isten neue In­for­ma­tio­nen ver­mit­teln—die Verbindung selbst ist möglicher­weise nicht offen­sichtlich. Wenn aber keine Beson­der­heiten fol­gen, ist der Satz wahrschein­lich ein Ap­plaus-Licht.

Ich bin ver­sucht, ein­mal einen Vor­trag zu halten, der nur aus Ap­plaus-Lichtern besteht, und zu se­hen, wie lange es dauert, bis das Publikum an­fängt zu lachen:

Ich möchte Ih­nen heute vorschla­gen, die Risiken und Chan­cen fortschrit­tlicher kün­stlicher In­tel­li­genz in Ein­klang zu bringen. Wir sol­lten die Risiken ver­mei­den und, soweit möglich, die Chan­cen wahrnehmen. Wir sol­lten uns nicht un­nötig mit un­nöti­gen Ge­fahren au­seinan­der­set­zen. Um diese Ziele zu er­re­ichen, müssen wir vernün­ftig und ra­tio­nal pla­nen. Wir sol­lten nicht in Furcht und Panik han­deln oder der Techno­pho­bie nachgeben. aber wir sol­lten auch nicht mit blin­der Begeisterung han­deln. Wir sol­lten die In­ter­essen aller Parteien re­spek­tieren, die an der Sin­gu­lar­ität beteiligt sind. Wir müssen sich­er­stel­len, dass die Vorteile fortschrit­tlicher Tech­nolo­gien so vie­len Men­schen wie möglich zur Verfü­gung ste­hen, anstatt auf einige beschränkt zu sein. Wir müssen ver­suchen, gewalt­tätige Kon­flikte mit diesen Tech­nolo­gien so weit wie möglich zu ver­mei­den. und wir müssen ver­hin­dern, dass mas­sive zer­störerische Fähigkeiten in die Hände von In­di­vi­duen fallen. Wir sol­lten diese Fra­gen vorher durch­denken, nicht danach, es ist zu spät, um et­was dage­gen zu un­ternehmen …

[xSciFix]

I am tempted to give a talk some­time that con­sists of noth­ing but ap­plause lights, and see how long it takes for the au­di­ence to start laugh­ing:

I’m re­minded of your Tom Rid­dle a bit heh.

I think “a speech that con­sists of noth­ing but ap­plause lights” pretty much ap­plies to 99% of poli­ti­cal dis­course these days and in­stead be­ing amused at how long it takes the au­di­ence to re­al­ize you’d be em­bit­tered at how se­ri­ously ev­ery­one took the whole ex­er­cise. Maybe I have some bias to sort out but I think the ac­tual con­tent of what is be­ing said of­ten mat­ters very lit­tle to most peo­ple, as long as you hit the right buz­zwords and look con­vinc­ing/​con­fi­dent.

[Omeg]

I am here to pro­pose to you to­day that we should not bal­ance the risks and op­por­tu­ni­ties of ad­vanced ar­tifi­cial in­tel­li­gence. We should wel­come the risks and re­main blind to op­por­tu­ni­ties. We should need­lessly con­front en­tirely un­nec­es­sary dan­gers. To achieve these goals, we must plan stupidly and ir­ra­tionally. We should act in fear and panic, and give in to techno­pho­bia; al­ter­na­tively, we should act in blind en­thu­si­asm. We should only re­spect the in­ter­ests of some par­ties with a stake in the Sin­gu­lar­ity. We must try to en­sure that the benefits of ad­vanced tech­nolo­gies re­main re­stricted to a small num­ber of peo­ple, rather than ac­crue to as many in­di­vi­d­u­als as pos­si­ble. We must en­courage, even if it’s im­pos­si­ble, vi­o­lent con­flicts us­ing these tech­nolo­gies; and we must see that the mas­sive de­struc­tive ca­pa­bil­ity falls into the hands of in­di­vi­d­u­als. We should think through these is­sues later, when it is too late to do any­thing about them . . .

I like those re­ver­sals tests. They are not only use­ful, but also quite hilar­i­ous.

[astralbrane]

“do what­ever the ma­jor­ity says” is not democ­racy.

True democ­racy finds the solu­tion that max­i­mizes the util­ity of all the vot­ers, not max­i­miz­ing the util­ity of half while com­pletely ig­nor­ing the other half.

[jeronimo196]

As do true com­mu­nism. Has there ever been such a democ­racy? How did it found out the util­ity for all its vot­ers? Sounds to me like a “no true Scots­man” en­com­pass­ing all known gov­ern­ment sys­tems. I think you should mod­ify your state­ment to some­thing like “A democ­racy should try and pro­tect the in­ter­ests of its minori­ties, as well as those of the ma­jor­ity.”