Adversarial System Hats

In Re­ply to: Ra­tion­al­iza­tion, Epistemic Hand­wash­ing, Selec­tive Processes

Eliezer Yud­kowsky wrote about sci­en­tists defend­ing pet hy­pothe­ses, and pros­e­cu­tors and defen­ders as ex­am­ples of clever ra­tio­nal­iza­tion. His pri­mary fo­cus was ad­vice to the well-in­ten­tioned in­di­vi­d­ual ra­tio­nal­ist, which is ex­cel­lent as far as it goes. But Anna Sala­mon and Steve Ray­hawk ask how a so­cial sys­tem should be struc­tured for group ra­tio­nal­ity.

The ad­ver­sar­ial sys­tem is widely used in crim­i­nal jus­tice. In the le­gal world, roles such as Prose­cu­tion, Defense, and Judge are all guaran­teed to be filled, with roughly the same amount of hu­man effort ap­plied to each side. Sup­pose in­di­vi­d­u­als chose their own roles. It is pos­si­ble that one role turns out more pop­u­lar. Be­cause differ­ent effort is ap­plied to differ­ent sides, se­lect­ing for the po­si­tions with the strongest ar­gu­ments will no longer much se­lect for po­si­tions that are true.

One role might be more pop­u­lar be­cause of an in­for­ma­tion cas­cade: in­di­vi­d­u­als read the ex­tant ar­gu­ments and then choose a role, striv­ing to al­ign them­selves with the truth, and cre­ate ar­gu­ments for that po­si­tion. Alter­nately, a role may be pop­u­lar due to sta­tus-based af­fili­a­tion, or striv­ing to be on the “win­ning” side.

I’m well aware that there are vastly more than two sides to most ques­tions. Imag­ine a list of ra­tio­nal­ist roles some­thing like IDEO’s “Ten Faces”.

Ex­am­ple ra­tio­nal­ist roles, leav­ing the ob­vi­ous ones for last:

  • The Me­di­a­tor, who strives for com­mon un­der­stand­ing and com­bin­ing ev­i­dence.

  • The Wise, who may not take a stand, but only crit­i­cize in­ter­nal con­sis­tency of ar­gu­ments.

  • The Per­pen­dicu­larist, who strives to break up po­lariza­tion by “pul­ling the rope side­ways”.

  • The Ad­vo­cate, who cham­pi­ons a con­tro­ver­sial claim or pro­posed ac­tion.

  • The De­trac­tor, who points out flaws in the con­tro­ver­sial claim or pro­posed ac­tion.

Due to nat­u­ral group phe­nom­ena (cas­cades, af­fili­a­tion), in or­der to achieve group ra­tio­nal­ity, there need to be so­cial struc­tures that strive to pre­vent those nat­u­ral phe­nom­ena. Roles might help.