Outline of Metarationality, or much less than you wanted to know about postrationality

There was a re­cent dis­cus­sion on Face­book that led to an ask for a de­scrip­tion of pos­tra­tional­ity that isn’t framed in terms of how it’s differ­ent from ra­tio­nal­ity (or rather per­haps more a challenge that such a thing could not be pro­vided). I’m ex­tra busy right now un­til at least the end of the year so I don’t have a lot of time for philos­o­phy and AI safety work, but I’d like to re­spond with at least an out­line of a con­struc­tive de­scrip­tion of post/​meta-ra­tio­nal­ity. I’m not sure ev­ery­one who iden­ti­fies as part of the meta­ra­tional­ity move­ment would agree with my con­struc­tion, but this is what I see as the core of our stance.

Fun­da­men­tally I think the core be­lief of meta­ra­tional­ity is that epistemic cir­cu­lar­ity (a.k.a. the prob­lem of the crite­rion, the prob­lem of per­cep­tion, the prob­lem of find­ing the uni­ver­sal prior) ne­ces­si­tates meta­phys­i­cal spec­u­la­tion, viz. we can’t re­li­ably say any­thing about the world and must in­stead make one or more guesses to over­come at least es­tab­lish­ing the crite­rion for as­sess­ing truth. Fur­ther, since the crite­rion for know­ing what is true is un­re­li­ably known, we must be choos­ing that crite­rion on some other ba­sis than truth, and so in­stead view that prior crite­rion as com­ing from use­ful­ness to some pur­pose we have.

None of this is rad­i­cal; it’s in fact all fairly stan­dard philos­o­phy. What makes meta­ra­tional­ity what it is comes from the deep in­te­gra­tion of this in­sight into our wor­ld­view. Rather than truth or some other crite­ria, telos (use­ful­ness, pur­pose) is the high­est value we can serve, not by choice, but by the trap of liv­ing in­side the world and try­ing to un­der­stand it from ex­pe­rience that is nec­es­sar­ily tainted by it. The rest of our wor­ld­view falls out of up­dat­ing our maps to re­flect this core be­lief.

To say a lit­tle on this, when you re­al­ize the pri­macy of telos in how you make judg­ments about the world, you see that you have no rea­son to priv­ilege any par­tic­u­lar as­sess­ment crite­rion ex­cept in so far as it is use­ful to serve a pur­pose. Thus, for ex­am­ple, ra­tio­nal­ity is im­por­tant to the pur­pose of pre­dict­ing and un­der­stand­ing the world of­ten be­cause we, through ex­pe­rience, come to know it to be cor­re­lated with mak­ing pre­dic­tions that later hap­pen, but other crite­ria, like com­pel­ling­ness-of-story and will­ing­ness-to-life, may be bet­ter drivers in terms of cre­at­ing the world we would like to later find our­selves in. For what it’s worth, I think this is the fun­da­men­tal dis­agree­ment with ra­tio­nal­ity: we say you can’t priv­ilege truth and since you can’t it some­times works out bet­ter to fo­cus on other crite­ria when mak­ing sense of the world.

So that’s the con­struc­tive part; why do we tend to talk so much about pos­tra­tional­ity by con­trast­ing it with ra­tio­nal­ity? I think two rea­sons. One, pos­tra­tional­ity is etiolog­i­cally tied to ra­tio­nal­ity: the ideas come from peo­ple who first went deep on ra­tio­nal­ity and even­tu­ally saw what they felt were limi­ta­tions of that wor­ld­view, thus we nat­u­rally tend to think in terms of how we came to the pos­tra­tional­ist wor­ld­view and want to show oth­ers how we got here from there. Se­cond and re­lat­edly, meta­ra­tional­ity is a wor­ld­view that comes from a change in a per­son that many of us choose to iden­tify with Ke­gan’s model of psy­cholog­i­cal de­vel­op­ment, speci­fi­cally the 4-to-5 tran­si­tion, thus we think it’s mainly worth­while to ex­plain our ideas to folks we’d say are in the 4/​ra­tio­nal­ist stage of de­vel­op­ment be­cause they are the ones who can di­rectly tran­si­tion to 5/​meta­ra­tional­ity with­out need­ing to go through any other stages first.

Feel free to ask ques­tions for clar­ifi­ca­tion in the com­ments; I have limited en­ergy available for ad­dress­ing them but I will try my best to meet your in­quiries. Also, sorry for no links; I wouldn’t have writ­ten this if I had to add all the links, so you’ll have to do your own googling or ask for clar­ifi­ca­tion if you want to know more about some­thing, but know that ba­si­cally ev­ery weird turn of phrase above is an in­vi­ta­tion to learn more.