Writes Putanumonit.com and SecondPerson.dating. @yashkaf on Twitter.
Jacob Falkovich
Off the top of my head, here are some new things it adds:
1. You have 3 ways of avoiding prediction error: updating your models, changing your perception, acting on the world. Those are always in play and you often do all three in some combination (see my model of confirmation bias in action).
2. Action is key, and it shapes and is shaped by perception. The map you build of any territory is prioritized and driven by the things you can act on most effectively. You don’t just learn “what is out there” but “what can I do with it”.
3. You care about prediction over the lifetime scale, so there’s an explore/exploit tradeoff between potentially acquiring better models and sticking with the old ones.
4. Prediction goes from the abstract to the detailed. You perceive specifics in a way that aligns with your general model, rarely in contradiction.
5. Updating always goes from the detailed to the abstract. It explains Kuhn’s paradigm shifts but for everything — you don’t change your general theory and then update the details, you accumulate error in the details and then the general theory switches all at once to slot them into place.
6. In general, your underlying models are a distribution but perception is always unified, whatever your leading model is. So when perception changes it does so abruptly.
7. Attention is driven in a Bayesian way, to the places that are most likely to confirm/disconfirm your leading hypothesis, balancing the accuracy of perceiving the attended detail correctly and the leverage of that detail to your overall picture.
8. Emotions through the lens of PP.
9. Identity through the lens of PP.
10. The above is fractal, applying at all levels from a small subconscious module to a community of people.
The new strain has been confirmed in the US and the vaccine rollout is still sluggish and messed up, so the above are in effect. The trades I made so far are buying out-of-the-money calls on VXX (volatility) and puts on USO (oil) and JETS (airlines) all for February-March. I’ll hold until the market has a clear, COVID related drop or until these options all expire worthless and I take the cap gains write-off. And I’m HODLing all crypto although that’s not particularly related to COVID. I’m not in any way confident that this is wise/useful, but people asked.
I don’t think it was that easy to get to the saturated end with the old strain. As I remember, the chance of catching COVID from a sick person in your household was only around 20-30%, and at superspreader events it was still just a small minority of total attendees that were infected.
The VXX is basically at multi-year lows right now, so one of the following is true:
1. Markets think that the global economy is very calm and predictable right now.
2. I’m misunderstanding an important link between “volatility = unpredictability of world economics” and “volatility = premium on short-term SP500 options”.
Some options and their 1-year charts:
JETS—Airline ETFXLE—Energy and oil company ETF
AWAY—Travel tech (Expedia, Uber) ETF
Which would you buy put options on, and with what expiration?
Those are good points. I think competition (real and potential) is always at least worth considering in any question of business, and I was surprised the OP didn’t even mention it. But yes, I can imagine situations where you operate with no relevant competition.
But this again would make me think that pricing and the story you tell a client is strictly secondary to finding these potential clients in the first place. If they were the sort of people who go out seeking help you’d have competition, so that means you have to find people who don’t advertise their need. That seems to be the main thing the author doing and the value they’re providing: finding people who need recruitment help and don’t realize it.
This pricing makes sense if your only competition is your client just going at it by themselves, in which case you clearly demonstrate that you offer a superior deal. But job seekers have a lot of consultants/agencies/headhunters they can turn to and I’d imagine your price mostly depends on the competition. In the worst case, you not only lose good clients to cheaper competition, but get an adverse selection of clients who would really struggle to find a job in 22 weeks and so your services are cheap/free for them.
This statement for example:
> Motivating you to punish things is what that part of your brain does, after all; it’s not like it can go get another job!
I’m coming more from a predictive processing / bootstrap learning / constructed emotion paradigm in which your brain is very flexible about building high-level modules like moral judgment and punishment. The complex “moral brain” that you described is not etched into our hardware and it’s not universal, it’s learned. This means it can work quite differently or be absent in some people, and in others it can be deconstructed or redirected — “getting another job” as you’d say.I agree that in practice lamenting the existence of your moral brain is a lot less useful than dissolving self-judgment case-by-case. But I got a sense from your description that you see it as universal and immutable, not as something we learned from parents/peers and can unlearn.
P.S.
Personal bias alert — I would guess that my own moral brain is perhaps in the 5th percentile of judginess and desire to punish transgressors. I recently told a woman about EA and she was outraged about young people taking it on themselves to save lives in Africa when billionaires and corporations exist who aren’t helping. It was a clear demonstration of how different people’s moral brains are.
I’ve come across a lot of discussion recently about self-coercion, self-judgment, procrastination, shoulds, etc. Having just read it, I think this post is unusually good at offering a general framework applicable to many of these issues (i.e., that of the “moral brain” taking over). It’s also peppered with a lot of nice insights, such as why feeling guilty about procrastination is in fact moral licensing that enables procrastination.
While there are many parts of the posts that I quibble with (such as the idea of the “moral brain” as an invariant specialized module), this post is a great standalone introduction and explanation of a framework that I think is useful and important.
But if evidence of that regrettable night is all over the internet, that is much worse. You then likely have a lot of other regrettable nights. College acceptances are rescinded, jobs lost.
I have a major quibble with this prediction. Namely my model is that the regrettability of nights, and moral character of people, is always graded on a curve, not absolutely.
Colleges still need to admit students. Employers still need employees. In a world where everyone smokes weed in high school but this is known about only 5% of students, it makes sense for jobs and colleges to exclude weed-smokers. But if 80% of people are known to have smoked weed (or had premarital sex, or shoplifted from CVS, or gotten into a fight), then it stops being a big deal.
An example from the other side would be cheating on your spouse: by some accounts half of us do it, but a lot fewer than half are publicly exposed for it. So today this still carries a huge stigma, but in a world where every cheater was being blackmailed, one of the main effects would be that cheating on a spouse would cease to be seen as an irredeemable sin.
The GNW theory has been kicking about for at least two decades, and this book has been published in 2014. Given this it is almost shocking that the idea wasn’t written up on LW before giving it’s centrality to any understanding of rationality. Shocking but perhaps fortunate, since Kaj has given it a thorough and careful treatment that enables the reader both to understand the idea and evaluate its merits (and almost certainly to save the purchase price of the book).
First, on GNW itself. A lot of the early writing on rationality used the simplified system 1 / system 2 abstraction as the central concept. GNW puts actual meat on this skeleton, describing exactly what unconscious (formerly known as system 1) processes can and can’t do, how they learn, and under what conditions consciousness comes into play. Kaj elaborates more on system 2 in another post, but this review offers enough to reframe the old model in GNW-terms — a reframing that I’ve been convinced is more accurate and meaningful.As for the post itself, it’s main strength and weakness is that it’s very long. The length is not due to fluff — I’ve compiled my own summary of this post in Roam that runs more than 1,000 words, with almost every paragraph worthy of inclusion. But perhaps, in particular for purposes of a book, the post could more fruitfully broken up in two parts: one to describe the GNW model and its implications, one to cover the experimental evidence for the model and its reliability. The latter takes up almost half of the text of the post by volume, and while it is valuable the former could perhaps stand alone as a worthwhile article (with a reference to a discussion of the experiments so people can assess whether they buy it).
D’oh. I’m dumb.
EDIT: The Treacherous Path was published in 2020 so never mind.
Thank you (and to alkjash) for the nomination!
I guess I’m not supposed to nominate things I wrote myself, but this post, if published, should really be read along with The Treacherous Path to Rationality. I hope someone nominates that too.
This post is an open invitation to everyone (such as the non-LWers who may read the books to join us). The obvious question is whether this actually works for everyone, and the latter post makes the case for the opposite-mood. I think that in conjunction they offer a much more balanced take on who and what applied rationality is good for.
Do you have trouble writing for short periods of time, or do you have enough long chunks of free time that there’s no use for small chunks?
If my life was so busy that I couldn’t even find 4-5 hourlong chunks throughout the week I probably wouldn’t blog at all. I sometimes write in 15-20 minute bits while in the office (remember those?) but almost every single post took a multi-hour chunk to come together.
Yes, really smart domain experts were smarter and earlier but, as you said, they mostly kept it to themselves. Indeed, the first rationalists picked up COVID worry from private or unpublicized communication with domain experts, did the math and sanity checks, and started spreading the word. We did well on COVID not by outsmarting domain experts, but by coordinating publicly on what domain experts (especially any with government affiliations) kept private.
We didn’t get COVID, for starters. I live in NYC, where approximately 25% of the population got sick but no rationalists that I’m aware of did.
If I, a rationalist atheist, was in Francis Bacon’s shoes I would 100% live my life in such a way that history books would record me as being a “devout Anglican”.
The longer (i.e., more iterations) you spend in the shaded triangles of defection the more you’ll be pulled to the defect-defect equilibrium as a natural reaction to what the other person is doing and the outcome you’re getting. The longer you spend in the middle “wedge of cooperation”, the more you’ll end moving up and to the right in Pareto improvements. So we want to make that wedge bigger.
The size of that wedge is determined by the ratio of a player’s outcome from C-C to their outcome in D-D. In this case the ratio is 2:1, so the wedge is between the slopes of 2 and 1⁄2. If C-C only guaranteed 1.1-1.1 to each player while a defection got them at least 1, the wedge would be a tiny sliver. Conversely, if the payoff for C-C was 999-999 almost the entire square would be the wedge.
But the bigger the wedge, the more difference there is between outcomes on the pareto frontier so the outcome of 100% C-C is a lot less stable than if any deviation from it immediately led to non-equilibrium points that degenerate to D-D.
Here’s what I wrote about coordinated moving when Raymond was talking about leaving the Bay for a while:
“Coordinated moving seems hard. It seems unlikely to happen. But, I think that uncoordinated moving can end up quite coordinated.
If I’m thinking of leaving Brooklyn, I have 10,000 small towns to choose from. If [Zvi, or Ray, or anyone like that] publicizes which one he goes to after doing research, that town is immediately in my top 10 options I’ll actually consider. Not just because I’d want to live near [Zvi/Ray] and I trust his research, but also because I know that hundreds of other people I like would know about that town and consider moving there. So if people just move out without coordinating but tell all their friends about it, I think we’ll end up with decent enough agglomerations of friends wherever the pioneers end up going.”
On a related note, I’m planning to go on a small road trip around the northeast in July and would love to visit you in Warwick if you’re accepting visitors (got tested this week, alas no antibodies, still distancing at home).
There’s a whole lot to respond to here, and it may take the length of Surfing Uncertainty to do so. I’ll point instead to one key dimension.
You’re discussing PP as a possible model for AI, whereas I posit PP as a model for animal brains. The main difference is that animal brains are evolved and occur inside bodies.
Evolution is the answer to the dark room problem. You come with prebuilt hardware that is adapted a certain adaptive niche, which is equivalent to modeling it. Your legs are a model of the shape of the ground and the size of your evolutionary territory. Your color vision is a model of berries in a bush, and your fingers that pick them. Your evolved body is a hyperprior you can’t update away. In a sense, you’re predicting all the things that are adaptive: being full of good food, in the company of allies and mates, being vigorous and healthy, learning new things. Lying hungry in a dark room creates a persistent error in your highest-order predictive models (the evolved ones) that you can’t change.
Your evolved prior supposes that you have a body, and that the way you persist over time is by using that body. You are not a disembodied agent learning things for fun or getting scored on some limited test of prediction or matching. Everything your brain does is oriented towards acting on the world effectively.
You can see that perception and action rely on the same mechanism in many ways, starting with the simple fact that when you look at something you don’t receive a static picture, but rather constantly saccade and shift your eyes, contract and expand your pupil and cornea, move your head around, and also automatically compensate for all of this motion. None of this is relevant to an AI who processes images fed to it “out of the void”, and whose main objective function is something other than maintaining homeostasis of a living, moving body.
Zooming out, Friston’s core idea is a direct consequence of thermodynamics: for any system (like an organism) to persist in a state of low entropy (e.g. 98°F) in an environment that is higher entropy but contains some exploitable order (e.g. calories aren’t uniformly spread in the universe but concentrated in bananas), it must exploit this order. Exploiting it is equivalent to minimizing surprise, since if you’re surprised there some pattern of the world that you failed to make use of (free energy).
Now if you just apply this basic principle to your genes persisting over an evolutionary time scale and your body persisting over the time scale of decades and this sets the stage for PP applied to animals.
For more, here’s a conversation between Clark, Friston, and an information theorist about the Dark Room problem.