Public Positions and Private Guts [Transcript]

(Talk given on Sun­day 21st June, over a zoom call with 40 at­ten­dees. Vaniver is re­spon­si­ble for the talk, ja­cob­ja­cob is re­spon­si­ble for the tran­scrip­tion)

Ben Pace: Thank you ev­ery­one very much for com­ing. All 41 of us. This is a LessWrong event. So it is less wrong than your nor­mal events. Ja­cob and I wanted to try out some on­line events, see what was fun. We pinged a bunch of the cu­rated au­thors who write great stuff and said, “Do you also want to give a short talk?” And a bunch of them were like, “Oh that sounds nice, to ac­tu­ally see the peo­ple who read my stuff, rather than just imag­in­ing them.”

Ben Pace: So, we’re go­ing to have five minute talks. I’ll keep time. And then we’ll have some Q&A af­ter­wards, max­i­mum 10 min­utes but shorter. And there’s go­ing to be five talks. Vaniver, if you’d like to be­gin?


Vaniver: Cool! Hi, I’m Vaniver. The thing I’m go­ing to be talk­ing about is Public Po­si­tions and Pri­vate Guts, a blog post that I wrote a while ago that got cu­rated. It’s origi­nally due to a se­ries of talks given by Anna Sala­mon at some work­shops.

Vaniver: So, what is this about? Why do we care?

Vaniver: Well, there’s some things that hap­pen some­times, like start-up founders who have an idea that they strongly an­ti­ci­pate will work, but they can’t ex­plain it to other peo­ple, they can’t prove that the thing will work. If they could prove it, it wouldn’t be a start-up any­more, it would already be some ma­ture busi­ness some­where.

Vaniver: Similarly, you will run across peo­ple who are PhD stu­dents who have this log­i­cal, air­tight ar­gu­ment that they should get their PhD, and yet they’re mys­te­ri­ously un­in­ter­ested in do­ing any work on their dis­ser­ta­tion. And so there’s this ques­tion of, what’s up with that? Why isn’t there this one map of the world that goes both ways?

Vaniver: So there are these two clusters of knowl­edge. I’m go­ing to talk first about a sort of com­mu­ni­ca­tion, that I think defines these clusters. For that I’m go­ing to talk a bit about for­mal com­mu­ni­ca­tion. Ba­si­cally, in philos­o­phy, there’s this model of how peo­ple talk to each other; you have a shared con­text where both the speaker and the au­di­ence know all the things in the shared con­text. The speaker will add ad­di­tional facts, one at a time (like maybe I’ve ob­served a thing that peo­ple don’t know about yet). And, also, log­i­cal facts count; if there’s, A in the con­text and also “if A, then B”, as­sert­ing B is a thing that might not have been in the con­text yet be­cause the au­di­ence isn’t log­i­cally om­ni­scient.

Vaniver: And so, one of the facts about this pro­cess is, if each of these ob­ser­va­tions doesn’t con­tra­dict the things that’s already in the shared con­text, and is triv­ially check­able in this one step, you can end up be­liev­ing the things that come out at the end, much like a math proof and that sort of thing, at least as much as you trust the con­text that you started off with.

Vaniver: And so an in­ter­est­ing fact about the sort of things you can fit into this com­mu­ni­ca­tion style is they all have to be eas­ily jus­tifi­able. If I have a point that I want to get across that re­quires five differ­ent com­pli­cated ar­gu­ments to sup­port it, un­less I can go through each one of these com­pli­cated ar­gu­ments in a se­rial fash­ion, you’re not go­ing to be able to build this thing us­ing this for­mal com­mu­ni­ca­tion style.

Vaniver: And so, Public Po­si­tions are these sorts of be­liefs that have been op­ti­mized for jus­tifi­a­bil­ity or pre­sen­ta­tion. The PhD stu­dent that has this log­i­cal ar­gu­ment that they have worked through with other peo­ple on why they should get their PhD, they have this pub­lic po­si­tion, they have this for­mal com­mu­ni­ca­tion to back it up.

Vaniver: Pri­vate Guts, in con­trast to this, they’re not mu­tu­ally ex­clu­sive to Public Po­si­tions, but they’re defined in a differ­ent way for a differ­ent pur­pose. They’re try­ing to ac­tu­ally an­ti­ci­pate things about the fu­ture. And they come from the ac­tual his­tor­i­cal causes of the be­lief. So, for ex­am­ple, this PhD stu­dent might his­tor­i­cally want the PhD be­cause their fam­ily always re­spected ed­u­ca­tion, and when they think about quit­ting the PhD, they can’t do it be­cause that would mean they’re a “quit­ter”.

Vaniver: When you think about train­ing a neu­ral net­work to rec­og­nize pic­tures of dogs and cats, it will use lots of lit­tle pieces of in­for­ma­tion to come to its con­clu­sion, in a way that’s opaque and difficult to un­der­stand be­cause it’s not op­ti­miz­ing at all for un­der­stand­abil­ity, it’s just op­ti­miz­ing for the suc­cess met­ric of, did it cor­rectly an­ti­ci­pate the thing or not?

Vaniver: And so, the startup founder’s com­pli­cated rea­son for be­liev­ing that their startup will work comes from this sort of thing. There’s lots of lit­tle pieces that all fit to­gether in their mind, but they can’t eas­ily ex­plain it or else this would be a wide­spread be­lief that the startup would work.

Vaniver: So, any­way, a lot of CFAR re­lated things re­late to how to build bridges be­tween these two sorts of things so that peo­ple who are con­vinced of some­thing through a log­i­cal ar­gu­ment also end up feel­ing it in their guts. And also, peo­ple who feel a thing in their guts, that they don’t have this for­mal po­si­tion for, are able to figure out how to draw out these many small pieces of data and con­struct some­thing that’s rea­son­able and ar­tic­u­la­ble.


Ben Pace: All right, that’s five min­utes. It sounded like ac­tu­ally you maybe just nat­u­rally stopped?

Vaniver: Just un­der the wire.

Ben Pace: Cool, cool. Thanks. So the PhD guy has a bunch of for­mal ar­gu­ments but not pri­vate guts for why he should do the thing, and he’s hav­ing a hard time trans­lat­ing be­tween those, and try­ing to have the for­mal ar­gu­ment in­form his pri­vate guts. And similarly, the startup guy is hav­ing a hard time nat­u­rally turn­ing the pri­vate guts into a for­mal­ized ar­gu­ment. That was what you said? That was ac­cu­rate?

Vaniver: Yeah, I think that’s my take on those ex­am­ples. I tried to come up with ex­am­ples that were a lit­tle differ­ent from the origi­nal his­tor­i­cal cause of this talk; which is some­thing like, many peo­ple who would take AI safety se­ri­ously with their speech but not with their ac­tual ac­tions. And there’s this ques­tion of, well why is that? And it’s like, oh it’s be­cause they don’t feel it com­ing for real, in the same way they might feel cli­mate change is com­ing for real, or some­thing.

Ben Pace: Yeah. Pa­trick, would you like to ask a ques­tion?

Pa­trick LaVic­toire: Build­ing on this, I have some­thing to say about how to model the in­ter­nal ex­pe­rience of this hap­pen­ing and what you can do about it. But I think, in­stead of a com­ment, I’m go­ing to talk for five min­utes about it and men­tion that your point is rele­vant to mine.

Vaniver: Cool!

Ben Pace: Sounds good. How does it tend to look when peo­ple suc­cess­fully turn their pri­vate guts into for­mal ar­gu­ments?

Vaniver: Yeah, so I think part of this is com­ing up with com­mu­ni­ca­tion styles that aren’t so much for­mal com­mu­ni­ca­tion. One thing that has grown more pop­u­lar over the last few years is this idea of do­ing dou­ble-crux, where you and an­other per­son will both try and look at your ac­tual be­lief sys­tem and say “this is the thing that would change my mind about this sub­ject that we dis­agree on”, and you jointly ex­plore this to­gether.

Vaniver: It’s in­ter­est­ing be­cause when you watch a pub­lic de­bate, of­ten you’ll find the things that are said are de­signed to con­vince you, the au­di­ence, or be broadly ap­pli­ca­ble. But when you watch a dou­ble-crux, this is the op­po­site of what they’re do­ing. They’re try­ing to fo­cus, laser-like, on what do I, Vaniver, care about in this is­sue? Even if only two per­cent of the au­di­ence cares about this par­tic­u­lar part of the is­sue, it’s the bit that’s crux-y for me.

Vaniver: So I think there’s a way in which for­mal com­mu­ni­ca­tion does ac­tu­ally limit the sort of things you can be­lieve. In the same way that be­ing re­lentlessly em­piri­cal about the world, in­stead of the­o­ret­i­cal, means that you can only be­lieve things that you’ve already seen hap­pen in the past in­stead of also be­liev­ing in things that you pre­dict will hap­pen in the fu­ture.

Ben Pace: Yeah, that makes sense. I’m just cu­ri­ous, have you also seen ex­am­ples of peo­ple turn­ing the ex­plicit for­mal ar­gu­ments into their pri­vate guts, and what that’s felt like?

Vaniver: Yeah. I think I’ve seen some ex­am­ples of that. The first thing that comes to mind is ac­tu­ally Robin Han­son’s con­strual level the­ory and the whole near/​far dis­tinc­tion. I think just hav­ing that in my men­tal vo­cab­u­lary, at least, it’s been much eas­ier to see what sort of be­liefs do I have that are just the color of my ban­ners, or some­thing, ver­sus what be­liefs do I have that are ac­tu­ally about an­ti­ci­pat­ing the fu­ture.

Vaniver: When I’ve come across some­thing that mat­ters to a lot of differ­ent facts of my life, like where I live, and I’m like, oh wait there’s this sort of home-town bias thing go­ing on here where liv­ing in this place is great be­cause it’s the place I live in. See­ing that sort of thing can help me switch to near mode and do much more of the “what are the ac­tual fac­tors that should mat­ter here? Are my guts linked up with the thing they should be linked up with?”

Ben Pace: Oh, a com­ment from Den­nis, which I think is a solid ques­tion about how this re­lates to Kah­ne­man’s Sys­tem 1 /​ Sys­tem 2. I think peo­ple of­ten think of Sys­tem 1 as the im­plicit one that has all the gears that are not eas­ily ac­cessible to my con­scious brain; my Sys­tem 2 is this sen­si­ble, ex­plicit rea­soner who jus­tifies his thoughts or some­thing.

Vaniver: Yeah, so I think they’re re­lated… I’m always a lit­tle hes­i­tant to say if some­thing is Sys­tem 1 or is Sys­tem 2 be­cause it’s this tech­ni­cal con­cept from psy­chol­ogy that I’m wary about get­ting wrong. But I do think there’s a way in which both Sys­tem 1 and Sys­tem 2 would be part of the pri­vate guts, where many of your Sys­tem 2 things, they’re de­liber­a­tive, they’re slow, but they’re not nec­es­sar­ily op­ti­mized for jus­tifi­ca­tion.

Vaniver: Similarly, when you look at pub­lic po­si­tions, I think the mode of it which I talked about, which is very much for­mal com­mu­ni­ca­tion-es­que, is very much this Sys­tem 2 things of, here’s my de­liber­a­tive rea­sons for the po­si­tion. But I think there’s an as­pect to pub­lic po­si­tions which is un­der­stand­ing the lay of the land, know­ing what the shared con­text is, know­ing what things will and won’t get you at­tacked or you will or won’t have to jus­tify. And that one feels like it’s of­ten very im­me­di­ate and re­ac­tive and in­tu­itive, and the var­i­ous other things that peo­ple say about Sys­tem 1.

Vaniver: I think there’s a big over­lap, but there’s also some bits on the di­ag­o­nals.

Ben Pace: That makes sense, yeah. Thanks very much, Vaniver. We’ll move onto the next one for now.