The Intelligent Social Web

Fol­low-up to: Fake Frame­works, Kenshō

Re­lated to: Slack, New­comblike Prob­lems are the Norm

Pre­vi­ously ti­tled: The Real-World Omega (see these com­ments)

I’d like to offer a fake frame­work here. It’s a lit­tle silly, and not fully jus­tified, but it keeps pro­duc­ing mean­ingful re­sults in my life when I use it. Some of my own per­sonal ex­am­ples are:

  • Over­com­ing a crush­ing depression

  • Learn­ing how to set aside my “perfor­mance mode” and be more au­then­tic and vuln­er­a­ble when I want to be

  • Shift­ing my at­tach­ment style from anx­ious-pre­oc­cu­pied to mostly secure

  • Fix­ing a lifelong prob­lem where I love ath­let­ics but I kept badly dam­ag­ing my body al­most any time I tried any­thing athletic

  • Set­ting my­self up to ex­pe­rience kenshō

I rec­og­nize that this doesn’t ad­dress Oli’s hes­i­ta­tion stem­ming from his sense of a lemons prob­lem. I’m afraid I don’t know how to, at least not yet. So in the mean­time, please take these as my re­ports of my ex­pe­rience and how it seems to me they came about, rather than as an at­tempt to per­suade.

Though I do hope y’all will benefit from this. If noth­ing else, I think this frame­work is fun. I’ve some­times de­scribed one use of it as “over­com­ing per­son­ally mean­ingful challenges by liv­ing an epic life.” It’s an awe­some frame­work to play with — es­pe­cially when oth­ers join in while re­mem­ber­ing that the frame­work is fake.

Here I’ll out­line a gen­eral the­ory I’ll want to call on in some up­com­ing posts. To­mor­row I’ll sep­a­rately post an ap­pli­ca­tion. (Origi­nally they were writ­ten as one post, but I got some feed­back on an ear­lier draft that con­vinced me they should be sep­a­rate.) At that point we should have some mean­ingful tools for wrestling with self-de­cep­tion.

So with that, let’s get started.


When you walk into an im­prov scene, you usu­ally have no idea what role you’re play­ing. All you have is some ini­tial prompt — some­thing like:

“You three are in a gar­den. The scene has to in­volve a stuffed bear some­how. Go!”

So now you’re look­ing to the other peo­ple there. Then some­one jumps for­ward and adds to the scene: “Oh, there it is! I’m glad we fi­nally found it!” Now you know a lit­tle bit about your char­ac­ter, and about the char­ac­ter of the per­son who spoke, but not enough to fully define any­one’s role.

You can then ex­pand the scene by adding some­thing: “It’s about time! We’re al­most late now.” Now you’ve speci­fied more about what’s go­ing on, who you are, and who the other play­ers are. But it’s still the case that none of you knows what’s go­ing on.

In fact, if you think you know, you’ll of­ten quickly be proven wrong. Maybe you imag­ine in that scene you’re an up­tight punc­tual per­son. And then the third per­son in the scene says to you, “What do you care, Alex? You’re always late to ev­ery­thing any­way!” Sur­prise! Now you need to flush who you thought you were from your mind, ac­cept the new frame, and run with it as part of your newly evolv­ing iden­tity. Other­wise the scene sort of crashes.

It would go more smoothly if you didn’t hold any pre­con­cep­tions about who you are or what’s go­ing on. The scene tends to work bet­ter if you stay in the pre­sent mo­ment and just jump in with the first thing that comes to mind (as long as it’s shaped by what has hap­pened so far). Then the col­lec­tion of in­ter­ac­tions and emerg­ing roles spon­ta­neously guides your be­hav­ior, which in turn help guide oth­ers’ be­hav­ior, all of which re­cur­sively defines the “who” and “what” of the scene. Your job as a player isn’t to play a char­ac­ter; it’s to co-cre­ate a scene.

We can sort of pre­tend that there’s a “di­rec­tor”: it’s the in­tel­li­gence that emerges be­tween the play­ers via their in­ter­ac­tions. It’s a dis­tributed sys­tem that com­putes re­la­tion­ships and con­text by guid­ing each node in its net­work to act freely within con­straints. From this van­tage point, the net­work guides play­ers, and the job of each player is to be guid­able but not purely pas­sive (since a pas­sive node is just re­lay­ing in­for­ma­tion rather than aid­ing in the com­pu­ta­tion). As long as ev­ery­one in­volved is plugged into and re­spon­sive to this net­work, the scene will usu­ally play out well.


I sus­pect that im­prov works be­cause we’re do­ing some­thing a lot like it pretty much all the time. The web of so­cial re­la­tion­ships we’re em­bed­ded in helps define our roles as it forms and in­cludes us. And that same web, as the dis­tributed “di­rec­tor” of the “scene”, guides us in what we do.

A lot of (but not all) peo­ple get a strong hit of this when they go back to visit their fam­ily. If you move away and then make new friends and sort of be­come a new per­son (!), you might at first think this is just who you are now. But then you visit your par­ents… and sud­denly you feel and act a lot like you did be­fore you moved away. You might even try to hold onto this “new you” with them… and they might re­spond to what they see as strange be­hav­ior by try­ing to nudge you into act­ing “nor­mal”: ig­nor­ing sur­pris­ing things you say, chang­ing the topic to some­thing fa­mil­iar, start­ing an old fight, etc.

In most cases, I don’t think this is mal­ice. It’s just that they need the scene to work. They don’t know how to in­ter­act with this “new you”, so they tug on their con­nec­tion with you to pull you back into a role they rec­og­nize. If that fails, then they have to re­define who they are in re­la­tion to you — which of­ten (but not always) hap­pens even­tu­ally.

I’m ba­si­cally tak­ing as an ax­iom of this frame­work that peo­ple need the “scene” to work — which is to say, they need to be able to play out their roles in re­la­tion to oth­ers’ roles within a co­her­ent con­text. I don’t think why this is the case is rele­vant for us­ing this frame­work… but I’ll wave my hands at a vague just-so story any­way for the sake of pump­ing in­tu­ition: hu­man be­ings’ main sur­vival strat­egy seems to be based on co­or­di­nat­ing in of­ten com­plex ways in tribes. For the in­di­vi­d­ual, this means that fit­ting in be­comes paramount. For the group, this means know­ing what to ex­pect from each per­son is crit­i­cal. So a trade be­comes pos­si­ble: the in­di­vi­d­ual can fit into and benefit from the group as long as they’re play­ing a role that fits well with the col­lec­tive.

This can re­sult in some pretty strange roles. From this van­tage point, a per­son who re­peat­edly leaves one abu­sive re­la­tion­ship only to get into an­other roughly similar one ac­tu­ally makes a lot of sense: this is a role that this per­son knows how to play. It’s hor­rible, but it’s still bet­ter than not fit­ting into the so­cial scene. It cre­ates a co­her­ent re­la­tion­ship with some­one who’s will­ing to (or has to) play an “abuser” role, and of­ten with peo­ple in “res­cuer” roles too. The trap they’re in isn’t (just) that their cur­rent abu­sive part­ner is gaslight­ing or threat­en­ing them; it’s that they don’t have an­other role they can see how to play. Un­less and un­til that per­son finds a differ­ent one that fits into the so­cial web, the strands of that web will tug them back into their old role. They don’t have enough slack in the web around them to change their fate.

The same kind of web/​slack dy­nam­ics show up in more pleas­ant-to-play roles too. The priv­ilege of a mid­dle-class Amer­i­can white man by de­fault has him play­ing out some kind of roughly known story-like path (prob­a­bly in­volv­ing col­lege and hav­ing kids and maybe a di­vorce) that, in the end, will prob­a­bly still leave him be­ing one of the rich­est peo­ple on Earth. And all the while, he might well have no clue that he has other op­tions or even that he’s on a path — but he’ll still know, some­how, not to step off that path (“I have to go to col­lege; are you crazy?”). Never mind that his lack of slack here is awfully con­ve­nient for him.

I’ve watched re­li­gious con­ver­sions and de­con­ver­sions hap­pen via ba­si­cally the same mechanism. I knew a fel­low many years ago (unattached to this com­mu­nity) who was a proud athe­ist. Then he started dat­ing a Chris­tian girl. Some­thing like a month later, he started quot­ing the Bible — but “only be­cause they’re handy metaphors” and not be­cause he re­ally be­lieved any of that stuff, you see. It later turned out he’d been go­ing to church with her. He kept offer­ing rea­sons that seemed vaguely plau­si­ble (“It’s a neat group of peo­ple, and it mat­ters to her, and I can take the time to read”), but there’s a pat­tern here that was ob­vi­ous. A few months later he told me he’d con­verted. Last I heard they had moved to Utah.

The great part is, I knew this was go­ing to hap­pen when they started dat­ing. Why? Be­cause when I warned him that he might find him­self want­ing to be­lieve her re­li­gion once they started hav­ing sex, his re­ac­tion was to re­as­sure me by act­ing con­fi­dent that he was im­mune to this. That meant he was more fo­cused on man­ag­ing my per­cep­tion of him than he was in notic­ing how the so­cial web was tug­ging him to­ward a tran­si­tion of roles. I didn’t know if they’d stay to­gether, but I was pretty sure that if they did, he’d con­vert.

I could give liter­ally hun­dreds of ex­am­ples like this. From where I’m stand­ing, it looks like one of the great challenges of ra­tio­nal­ity is that peo­ple change their minds about mean­ingful things mostly only when the web tugs them into a new role. Ac­tu­ally think­ing in a way that for real changes your mind in ways that defy your web-given role is so­cially de­viant, and there­fore per­son­ally dan­ger­ous, and there­fore some­thing you’re mo­ti­vated not to learn how to do.

Ah, but if we’re im­mersed in a cul­ture where sta­tus and be­long­ing are tied to chang­ing our minds, and we can sig­nal that we’re open to up­dat­ing our be­liefs, then we’re good… as long as we know Good­hart’s De­mon isn’t lurk­ing in the shad­ows of our minds here. But surely it’s okay, right? After all, we’re smart and we know Bayesian math, and we care about truth! What could pos­si­bly go wrong?

Another challenge here is that the part of us that feels like it’s think­ing and talk­ing is (usu­ally) analo­gous to a char­ac­ter in an im­prov scene. The play­ers know they’re in a scene, but the char­ac­ters they’re play­ing don’t. The char­ac­ters also aren’t sur­prised about who or what they are: the not-know­ing of iden­tity and con­text is some­thing only the play­ers ex­pe­rience, to open them­selves up to the guidance of the dis­tributed “di­rec­tor”. This means that (a) the char­ac­ters are ac­tively wrong about why they do what they do, and (b) they are also deeply con­fused about how much sense ev­ery­thing makes and don’t know they’re con­fused.

I claim that most of us, most of the time, are play­ing out char­ac­ters as defined by the sur­round­ing web — and we usu­ally haven’t a clue how to Look at this fact, much less in­ten­tion­ally use our web slack to change our sto­ries.

I think this is also part of why im­prov is challeng­ing: you have to set aside the char­ac­ter you would nor­mally play in or­der to cre­ate room for some­thing new.


There’s a way in which the so­cial web holds the po­si­tion of Omega in an on­go­ing set of New­comblike prob­lems. The web as a whole wants to know what kind of role you’re play­ing, and how well you’re go­ing to play it, so that it can know what to ex­pect of you. So, a lot of its dis­tributed re­sources go into com­put­ing a model of you.

One of the more ob­vi­ous trans­mis­sion meth­ods is chat — idle gos­sip, sto­ry­tel­ling, spec­u­la­tion, small talk. Peo­ple sync up their im­pres­sions of some­one they’ve met, and try to make sense of sur­pris­ing events in con­ver­sa­tion. If a lover brings their part­ner some flow­ers and the re­cip­i­ent freaks out and runs off, sud­denly there’s a need to un­der­stand, and the flower-giver might try ask­ing a mu­tual friend for some help un­der­stand­ing. And even if they do come to un­der­stand (“Oh, that’s be­cause their last part­ner brought them flow­ers to break up with them”), there’s of­ten an im­pulse to share the story with friends, so that the web as a whole can hold ev­ery­one in sen­si­ble roles and make the scene work. (“Oh, we had a funny mi­s­un­der­stand­ing ear­lier, poor Sam….”)

A lot of this is trans­mit­ted more sub­tly too, in body lan­guage and fa­cial ex­pres­sions and vo­cal tone and so on. If Bob is “creepy” (i.e., is play­ing a “creepy” role in the web), then it speaks vol­umes if ev­ery­one who meets Bob then cringes just a tiny bit when he’s later men­tioned even if they say only good things about him. This means that some­one who has never met Bob can get a “vibe” about him from mul­ti­ple peo­ple in a way that shapes how they in­ter­pret what Bob says and does when they fi­nally do meet him.

Some­times, some peo­ple with enough web-savvy weaponize this. It doesn’t mean any­thing for some­one to “be creepy” ex­cept that they have a web-like im­pact on oth­ers — which is to say, they have a “creepy” role. In a healthy net­work, this cor­re­lates with some­thing ac­tu­ally mean­ingfully bad that’s worth track­ing. But be­cause per­ceived roles shape what peo­ple ex­pect of a per­son, it’s enough for a ru­mor to echo through the web in or­der for some­one to be in­ter­preted as “creepy”. So a suffi­ciently cun­ning per­son could ac­tu­ally cause some­one to be slowly iso­lated and dis­trusted with­out there be­ing any facts at all to jus­tify this as Omega’s stance.

(And yes, I’ve seen this hap­pen. Many times.)

The same kind of thing can hap­pen with “pos­i­tive” la­bels, too. What it means for some­one to be fit for a lead­er­ship role, in Omega’s eyes, is that they are seen as com­pat­i­ble with that role. So if some­one is tall, at­trac­tive, and ei­ther vi­cious or strong de­pend­ing on how you choose to see it, it might be enough to have the “strong” in­ter­pre­ta­tion echo more pow­er­fully than the “vi­cious” one in or­der for the web to con­spire to put them in a lead­er­ship po­si­tion.

…which means that even peo­ple who are seen as good lead­ers might not, in fact, be good lead­ers in the sense of mak­ing good lead­er­ship de­ci­sions. But they are by defi­ni­tion good lead­ers in the sense of play­ing the role well. After all, if the gen­eral con­sen­sus is that Abra­ham Lin­coln was a great Pres­i­dent, then there’s a sense in which that makes it true, since that’s what “great” means here. The “ex­pla­na­tions” there­after are of­ten sto­ries to jus­tify one’s hold­ing of a pop­u­lar opinion.

The same thing holds for when some­one seems “ra­tio­nal”. This is one rea­son to worry deeply when mem­bers of sub­groups in­ter­nally agree with each other on who is a top-notch clear thinker or “re­ally a ra­tio­nal­ist” but dis­agree with peo­ple in other sub­groups. This looks less to me like peo­ple seek­ing truth, and a lot more like groups en­gag­ing in a sub­tle memetic bat­tle over what “ra­tio­nal” gets to mean.

From where I’m stand­ing, it looks to me like we’re all im­mersed in not-know­ing, while our “char­ac­ters” keep talk­ing as though they know what’s go­ing on, im­plic­itly fol­low­ing some hid­den-to-them script.


The web en­codes a lot of its guidance about what to ex­pect and how to be­have via the struc­ture of sto­ries. Or rather, story struc­tures are what ex­pec­ta­tions about roles and scenes are.

The trou­ble is, a lot of the sto­ries we talk about have the struc­ture of what our char­ac­ters are sup­posed to say rather than of what ac­tu­ally hap­pens. Imag­ine a movie where the new kid at a school gets bul­lied by the pop­u­lar kids and then makes friends with quirky out­casts. What hap­pens to the bul­lies in the end? In real life, bul­lies of­ten don’t get their come­up­pance — but hav­ing this fic­tional story in our hearts lets us play out the in­dig­nant ver­sions of our char­ac­ters in the real-life ver­sion. Be­cause the bul­lies aren’t sup­posed to get away with it, right? That wouldn’t be fair!

Some parts of our story-like in­tu­itions are scripts. Some are things our scripts say we should think or talk about. Some are merely in­ci­den­tal de­tails. Suss­ing out which parts are which is part of the trick of get­ting this frame­work to work for you. For in­stance, the stereo­typ­i­cal story of the wor­ried nag­ging wife con­fronting the emo­tion­ally dis­tant hus­band as he comes home re­ally late from work… is ac­tu­ally a pretty good car­i­ca­ture of a script that lots of cou­ples play out, as long as you know to ig­nore the gen­der and class as­sump­tions em­bed­ded in it.

But it’s hard to sort this out with­out just en­act­ing our scripts. The ver­sion of you that would be think­ing about it is your char­ac­ter, which (in this frame­work) can ac­cu­rately un­der­stand its own role only if it has enough slack to be­come genre-savvy within the web; oth­er­wise it just keeps play­ing out its role. In the hus­band/​wife script men­tioned above, there’s a ten­dency for the “wife” to get ex­cited when “she” learns about the re­la­tion­ship script, be­cause it looks to “her” like it sug­gests how to save the re­la­tion­ship — which is “her” en­act­ing “her” role. This of­ten ag­gra­vates the fears of the “hus­band”, caus­ing “him” to pull away and act dis­mis­sive of the script’s rele­vance (which is “his” role), driv­ing “her” to in­sist that they just need to talk about this… which is the same pat­tern they were in be­fore. They try to be­come genre-savvy, but there (usu­ally) just isn’t enough slack be­tween them, so the effort merely changes the topic while they play out their usual scene.

If you know how to Look at lived sto­ries, then I think a way out can start be­com­ing a lot more ob­vi­ous. Un­for­tu­nately, I don’t think I can de­scribe it very well for a gen­eral au­di­ence, be­cause how any­one re­ceives what I say is it­self sub­ject to the in­fluence of lived sto­ries. But if you can: try Look­ing in the pre­sent mo­ment at your own sense of not-know­ing, no­tice that the same thing is al­ive in oth­ers, and watch as the story arises and plays out across all of you.

To­mor­row I’ll share a weaker but eas­ier-to-use par­tial solu­tion that I think doesn’t re­quire Look­ing.


This was long, so I’ll try to sum­ma­rize:

  • You can choose to see so­cial groups at all scales as run­ning a dis­tributed com­pu­ta­tion across the so­cial web. If you view that pro­cess as gen­er­at­ing an in­tel­li­gent agent, you can think of this web-agent as the real-world Omega as it tries to pre­dict and guide each per­son’s be­hav­ior.

  • Omega offers each per­son a trade: pri­ori­tize mak­ing the scene work, and you’ll be in­cluded in it. In fact, Omega is the ag­gre­gate efforts of all the peo­ple who have ac­cepted that trade. And ba­si­cally ev­ery­one we know about ac­cepts this trade.

  • Every­thing about your­self that you have con­scious ac­cess to is sub­ject to your role as part of Omega. If you try to defy this, then your fate will play out through your defi­ance.

  • Room for in­ter­pre­ta­tion in your role in the scene means your script has room to change. This is slack in the so­cial web.

  • There’s a way of di­rectly see­ing how to change your fate by Look­ing. That’s not helpful un­less and un­til you learn how to Look, though.

I’ll close this post by not­ing that there’s a meta-level to track here. In the story The Em­peror’s New Clothes, the child’s ut­ter­ance wasn’t enough on its own to pop the illu­sion:

“But the Em­peror has noth­ing at all on!” said a lit­tle child.
“Listen to the voice of in­no­cence!” ex­claimed his father; and what the child had said was whispered from one to an­other.
“But he has noth­ing at all on!” at last cried out all the peo­ple. The Em­peror was vexed, for he knew that the peo­ple were right; but he thought the pro­ces­sion must go on now! And the lords of the bed­cham­ber took greater pains than ever, to ap­pear hold­ing up a train, al­though, in re­al­ity, there was no train to hold.

What if the father had in­stead re­sponded “No, child, you’re just too fool­ish to see his fine gar­ments”? He might have, out of fear of what those who were stand­ing nearby might think of him and his kid. Then the child’s sim­ple voice of rea­son would not be heard.

Or what if the peo­ple near the father/​child pair had felt too un­easy to pass along what the child had said?

What if the Em­peror could have in­stil­led this kind of ner­vous­ness in his peo­ple ahead of time? He might have thought that there will be in­no­cent chil­dren in the pa­rade, and it might have oc­curred to some part of him that they had best not be taken se­ri­ously — to spare oth­ers their em­bar­rass­ment, of course. Then, oh then what strange pro­pa­ganda they all would see.

Some of the scripts Omega as­signs work less well if they’re known. Be­cause of this, Omega will of­ten move to silence peo­ple who threaten to speak those frag­ile truths. This can show up, for in­stance, as peo­ple try­ing to dis­miss and dis­credit the per­son say­ing the idea rather than just the idea. The ar­gu­ments usu­ally sound sen­si­ble on the sur­face, but the un­der­ly­ing tone ring­ing through the strands of the web is “Don’t listen to this one.”

If it’s not clear why I’m men­tion­ing this, then I imag­ine it’ll be­come re­ally ob­vi­ous quite soon.