# abramdemski(Abram Demski)

Karma: 10,494
• The fol­low­ing ar­gu­ment comes from an in­tro so­ciol­ogy text:

If there are three peo­ple com­pet­ing, all of differ­ent strengths, it is worth­while for the two weak­est peo­ple to ban to­gether to defeat the strongest per­son. This takes out the largest threat. (Spe­cific game-the­o­retic as­sump­tions were not stated.)

Doesn’t this ba­si­cally ex­plain the phe­nomenon? If Zug kills Urk, I might be next! So I should ban to­gether with Urk to defeat Zug. Even if Urk doesn’t re­ward me at all for the help, my chances against Urk are bet­ter than my chances against Zug. (Un­der cer­tain as­sump­tions.)

• I agree, Yvain said it first, and it doesn’t sound like group se­lec­tion.

Con­cern­ing your group se­lec­tion com­ment, that does sound plau­si­ble… but be­ing rel­a­tively un­fa­mil­iar with tribal be­hav­ior, I would want to be sure that greedy genes were not spread­ing be­tween groups be­fore con­clud­ing that group se­lec­tion could ac­tu­ally oc­cur.

• Is this re­ally the case?

In fuzzy logic, one re­quires that the real-num­bered truth value of a sen­tence is a func­tion of its con­stituents. This al­lows the “solve it” re­ply.

If we swap that for prob­a­bil­ity the­ory, we don’t have that any­more… in­stead, we’ve got the con­straints im­posed by prob­a­bil­ity the­ory. The real-num­bered value of “A & B” is no longer a definite func­tion F(val(A), val(B)).

Maybe this is only a triv­ial com­pli­ca­tion… but, I am not sure yet.

• The crisp por­tion of such a self-refer­ence sys­tem will be equiv­a­lent to a Kripke fixed-point the­ory of truth, which I like. It won’t be the least fixed point, how­ever, which is the one I pre­fer; still, that should not in­terfere with the nor­mal math­e­mat­i­cal rea­son­ing pro­cess in any way.

In par­tic­u­lar, the crisp sub­set which con­tains only state­ments that could safely oc­cur at some level of a Tarski hi­er­ar­chy will have the truth val­ues we’d want them to have. So, there should be no com­plaints about the sys­tem com­ing to wrong con­clu­sions, ex­cept where prob­le­mat­i­cally self-refer­en­tial sen­tences are con­cerned (sen­tences which are as­signed no truth value in the least fixed point).

So; the ques­tion is: do the sen­tences which are as­signed no truth value in Kripke’s con­struc­tion, but are as­signed real-num­bered truth val­ues in the fuzzy con­struc­tion, play any use­ful role? Do they add math­e­mat­i­cal power to the sys­tem?

For those not fa­mil­iar with Kripke’s fixed points: ba­si­cally, they al­low us to use self-refer­ence, but to say that any sen­tence whose truth value de­pends even­tu­ally on its own truth value might be truth-value-less (ie, mean­ingless). The least fixed point takes this to be the case when­ever pos­si­ble; other fixed points may as­sign truth val­ues when it doesn’t cause trou­ble (for ex­am­ple, al­low­ing “this sen­tence is true” to have a value).

If dis­course about the fuzzy value of (what I would pre­fer to call) mean­ingless sen­tences adds any­thing, then it is by virtue of al­low­ing struc­tures to be defined which could not be defined oth­er­wise. It seems that adding fuzzy logic will al­low us to define “es­sen­tially fuzzy” struc­tures… con­cepts which are fun­da­men­tally ill-defined… but in terms of the crisp struc­tures that arise, cor­rect me if I’m wrong, but it seems fairly clear to me that noth­ing will be added that couldn’t be added just as well (or, bet­ter) by adding talk about the class of real-val­ued func­tions that we’d be us­ing for the fuzzy truth-func­tions.

To sum up: rea­son­ing in this way seems to have no bad con­se­quences, but I’m not sure it is use­ful...

• YKY,

The prob­lem with Kripke’s solu­tion to the para­doxes, and with any solu­tion re­ally, is that it still con­tains refer­ence holes. If I strictly ad­here to Kripke’s sys­tem, then I can’t ac­tu­ally ex­plain to you the idea of mean­ingless sen­tences, be­cause it’s always ei­ther false or mean­ingless to claim that a sen­tence is mean­ingless. (False when we claim it of a mean­ingful sen­tence; mean­ingless when we claim it of a mean­ingless one.)

With the fuzzy way out, the refer­ence gap is that we can’t have dis­con­tin­u­ous func­tions. This means we can’t ac­tu­ally talk about the fuzzy value of a state­ment: any claim “This state­ment has value X” is a dis­con­tin­u­ous claim, with value 1 at X and value 0 ev­ery­where else. In­stead, all we can do is get ar­bi­trar­ily close to say­ing that, by hav­ing con­tin­u­ous func­tions that are 1 at X and fall off sharply around X… this, I ad­mit, is rather nifty, but it is still a refer­ence gap. War­ri­gal refers to ac­tual val­ues when de­scribing the logic, but the logic it­self is in­ca­pable of do­ing that with­out run­ning into para­dox.

• “Should­ness” refers to a par­tic­u­lar very spe­cific way of pre­sent­ing the sys­tem’s >be­hav­ior, and it’s not free en­ergy. No­tice that you can de­scribe AI’s or man’s be­hav­ior >with phys­i­cal vari­a­tional prin­ci­ples as well, but that will have noth­ing to do with their >prefer­ence.

It seems to me that what SilasBarta is ask­ing for here is a defi­ni­tion of should­ness such that the above state­ment holds. Why is it in­valid to think that the sys­tem “wants” its physics? All you are in­di­cat­ing is that such is not what’s in­tended (which I’m sure SilasBarta knows)...

• I agree. The idea that low-en­tropy pock­ets that form are to­tally im­mune to a sim­plic­ity prior seems un­jus­tified to me. The uni­verse may be in a high-en­tropy state, but it’s still got phys­i­cal laws to fol­low! It’s not just do­ing things to­tally at ran­dom; that’s merely a con­ve­nient ap­prox­i­ma­tion. Maybe I am ig­no­rant here, but it seems like the prob­a­bil­ity of a par­tic­u­lar low-en­tropy bub­ble will be based on more than just its size.

• The rea­son is sim­ply that, in the mul­ti­ple wor­lds in­ter­pre­ta­tion, we do sur­vive—we just also die. If we ask “Which of the two will I ex­pe­rience?” then it seems to­tally valid to ar­gue “I won’t ex­pe­rience be­ing dead.”

• I agree with all of your sum­maries, so any read­ers should par­tially dis­count the many oc­cur­rences of “so far as I can tell.” :)

How do you re­spond to the claim that the few re­searchers who are work­ing di­rectly on very gen­eral, Solomonoff-ap­prox­i­mat­ing AI sys­tems are, when put to­gether, dan­ger­ous?

• This cry of “was it ever done any other way?” strikes me as his­tor­i­cally naive… ar­ranged mar­riages hap­pened, af­ter all, and still hap­pen. Dur­ing cer­tain space-time pe­ri­ods I un­der­stand it is/​was cus­tom­ary to have much younger brides than grooms, in which case it seems more rea­son­able to sur­prise rather than dis­cuss (since the groom may not have a great de­sire for the young bride’s opinions in the mat­ter).

In any case, it seems the ques­tion should be an­swered by a his­tor­i­cal so­ciol­o­gist...

• Sorry for the slow re­sponse. Solomonoff, un­til he died re­cently. Mar­cus Hut­ter is im­ple­ment­ing an AIXI ap­prox­i­ma­tion last I heard. Eray Ozku­ral, im­ple­ment­ing Solomonoff’s ideas. Sergey Pankov, im­ple­ment­ing an AIXI ap­prox­i­ma­tion.

# Al­gorithms as Case Stud­ies in Rationality

14 Feb 2011 18:27 UTC
38 points
• It’s an in­ter­est­ing ex­pe­rience to learn for­mal logic and then take a higher-level math class (any proof-in­ten­sive topic). Dur­ing the pro­cess of find­ing a proof, we ask all sorts of ques­tions of the form “does that im­ply that?”. How­ever, since we’re typ­i­cally prov­ing some­thing which we already know is a the­o­rem, we could log­i­cally an­swer: “Yes: any two true state­ments im­ply one an­other, and both of those state­ments are true.” This is a silly and un­helpful re­ply, of course. One way of see­ing why is to point out that al­though we may already be will­ing to be­lieve the the­o­rem, we are try­ing to con­struct an ar­gu­ment which could in­crease the cer­tainty of that be­lief; hence, the di­rec­tion of prop­a­ga­tion is to­wards the the­o­rem, so any be­lief we may have in the the­o­rem can­not be used as ev­i­dence in the ar­gu­ment.

What do you think, am I over­step­ping my bounds here? I feel like the prob­a­bil­is­tic case gives us some­thing more. In clas­si­cal logic, we ei­ther be­lieve the state­ment already or not; we don’t need to worry about count­ing ev­i­dence twice be­cause ev­i­dence is ei­ther to­tally con­vinc­ing or not con­vinc­ing at all.

• I ac­tu­ally cited the Wolfram ar­ti­cle be­cause I preferred it, but I went ahead and added a link to the wikipe­dia ar­ti­cle for those whose taste is closer to yours! Thanks.

The Risch al­gorithm for sym­bolic in­te­gra­tion is what first gave me a hunger to learn “the ac­tu­ally good ways of do­ing things” in this re­spect, and a sense that I might have to search for them be­yond the class­room. How­ever, I never did learn to use the Risch al­gorithm it­self! I don’t re­ally know whether it turns out to be good for hu­man use.

• Refer­ring to be­lief prop­a­ga­tion? The ac­tual pro­ce­dure does some­thing re­ally sim­ple: ig­nore the prob­lem. Ex­per­i­men­tally, this ap­proach has shown it­self to be very good in a lot of cases. Very lit­tle is known about what de­ter­mines how good an ap­prox­i­ma­tion this is, but if I re­call cor­rectly, it’s been proven that a sin­gle loop will always con­verge to the cor­rect val­ues; it’s also been proven that if all the lo­cal prob­a­bil­ity dis­tri­bu­tions are Gaus­sian, then the es­ti­mated means will also con­verge cor­rectly, but the var­i­ances might not.

Many things can be done to im­prove the situ­a­tion, too, but I’m not “up” on that at the mo­ment.

• I’ll sec­ond and con­straint-solv­ing al­gorithms; speci­fi­cally, the vari­able-or­der­ing heuris­tics seem helpful to me. Choose the vari­ables which most con­straint the prob­lem first! Note, the con­straint prop­a­ga­tion step is an­other in­stance of the sum-product al­gorithm. :)

• Al­gorithms I find use­ful that I didn’t put in the ar­ti­cle:

--Find de­cent solu­tions to pack­ing prob­lems by pack­ing the largest items first, then go­ing down in or­der of size

--Min­i­mum-de­scrip­tion-length ideas (no sur­prise to ra­tio­nal­ists! Just oc­cam’s ra­zor)

--Bi­nary search (just for find­ing a page in a book, but still, learn­ing the al­gorithm ac­tu­ally im­proved my speed at that task :p)

--Ex­plo­ra­tion vs ex­ploita­tion trade-off in re­in­force­ment learn­ing (I can’t say I’m sys­tem­atic about it, but learn­ing the con­cept made me aware that it is some­times ra­tio­nal to take ac­tions which seem sub­op­ti­mal from what I know, just to see what hap­pens)