Quinn
We need a name for the following heuristic, I think, I think of it as one of those “tribal knowledge” things that gets passed on like an oral tradition without being citeable in the sense of being a part of a literature. If you come up with a name I’ll certainly credit you in a top level post!
I heard it from Abram Demski at AISU′21.
Suppose you’re either going to end up in world A or world B, and you’re uncertain about which one it’s going to be. Suppose you can pull lever which will be 100 valuable if you end up in world A, or you can pull lever which will be 100 valuable if you end up in world B. The heuristic is that if you pull but end up in world B, you do not want to have created disvalue, in other words, your intervention conditional on the belief that you’ll end up in world A should not screw you over in timelines where you end up in world B.
This can be fully mathematized by saying “if most of your probability mass is on ending up in world A, then obviously you’d pick a lever L such that is very high, just also make sure that or creates an acceptably small amount of disvalue.”, where is read “the value of pulling lever L if you end up in world A”
critiques and complaints
I think one of the most crucial meta skills i’ve developed is honing my sense of who’s criticizing me vs. who’s complaining.
A criticism is actionable, implicitly often it’s from someone who wants you to win. A complaint is when you can’t figure out how you’d actionably fix something or improve based on what you’re being told.
This simple binary story is problematic. It can empower you to ignore criticism you don’t like by providing a set of excuses, if you’re not careful. Sometimes it’s operationally impossible to parse out a criticism that runs so deep that it unsettles your premises from a complaint! I think people who are building things can be excused for ignoring advice if the only actionable way of accepting that advice is to completely overhaul their approach, for reasons of focus and other logistical concerns. If it’s that rare time in a project when you are going back to the drawing board and starting over, that’s definitely time to mine complaints for useful insight.
Related: the legend of the amazon customer in the 90s who was insatiably filling out customer feedback forms, to the point where 2000s or 2010s amazon named a boardroom after him. The idea was that this guy helped them improve a lot—surely it would have been easy to dismiss him as a complainer, but they didn’t, they found actionable advice within the complaints. I think your ability to take something that isn’t intended to help you, isn’t actionable on it’s face, and mining it for actionable insight can be very important. But for filtering, for attention, for sanity, dismissing something quickly because it doesn’t seem like it can help you or the project improve can be valid as well.
It could be coincidental, but since then I think the rate of pondering founding/building ideas has increased. Perhaps my ability to see myself in a founder role has increased.
(Which isn’t specifically about profitable business models, so could be orthogonal to the billionaire suggestion: most of the “buildy” ideas I ponder are grant-reliant / unprofitable)
One time an EA just asked me “have you considered becoming a billionaire?”, which I found very potent.
I think of the 4 community reviews (5 if Yglesias is considered a community member), this is my favorite. I like that you carved right through wishing you were watching a different movie or expecting to watch a different movie, and just extracted the maximal value from the movie that you indeed actually watched. I feel like when I talk about ways the movie could’ve been better, I’m expressing dissatisfaction with the people who made it; when you do it, you’re suggesting ways for the characters to improve.
I like that what others interpreted as cheap shots at greedy or stupid villains you interpreted as a realistic portrayal of the way social reality commandeers situations.
When Isherwell didn’t take his technical concerns seriously, he should have gone and talked to the engineer in charge of the explosive sync
This calls to mind a genre I’d love to see exist—stories of competence or humility having to resort to being cunning to sneak around in mazes (in the sense of Zvi’s sequence) to get stuff done. Yes, in real life if a boss is too confident the move is to find somewhere on the org chart / chain of command someone who can be reasoned with. Some of us have been in large organizations, gotten results, and by that point it doesn’t matter if an idiot boss who if anything made it harder to succeed gets the credit. I don’t know a great citation for this, but the idea that top-down organizations have very real information flow and compression problems where people with decision-making power lack an inside view of what’s going on throughout the organization comes to mind.
They should have pushed for more redundant back-up plans to be enacted, even before they knew whether the first one worked or not.
Completely, 100%. In real life there’d be many more agents, many of whom with capital / ability to coordinate massive projects. This is probably the principal shortcoming of the movie’s realism, not counting various hard scifi comments about the real life state of the art in asteroid/comet deflection.
handed a miracle
The missed opportunity I’m most annoyed about is around this—In my version everything could go as planned via the first hypothesized miracle, then either 1. everyone could die anyway, or 2. they’d have to go back to the drawing board, get creative, decentralize (i.e. invest in other orgs or individuals), and try again. So much richer than the movie we actually watched.
hmu for a haskell job in decentralized finance. Super fun zero knowledge proof stuff, great earning to give opportunity.
Are shelling points the occam’s razor of mechanism design?
In game theory, a focal point (or Schelling point) is a solution that people tend to choose by default in the absence of communication. (wikipedia)
Intuitively I think simplicity is a good explanation for a solution being converged upon.
Does anyone have any crisp examples that violate the schelling point—occam’s razor correspondence?
Major guilty pleasure of mine is Aaron Sorkin, who once did a show called Newsroom about a large news broadcast project that, against all odds and incentives, doubles down on the duty of media elites to inform the public and so on. It’s either unbearably corny (insulting) or unbearably corny (affectionate) depending on who’s watching.
After the first broadcast of their re-invigorated show, the producer says
in the old days of like 10 minutes ago, we did the news well. You know how? We decided to.
I was thinking about this post and I got my streams crossed—the model of the JFK bit in my head accidentally inserted something like “we do this because we decide to”, and it worked really well! I find it motivating in the poetry sense to believe whatever illusion about agency or free will, especially at a collective level, that allows me to say “we are those who happened to step up” and flushing out any mention of reasons why we stepped up.
For some reason, “we decided to” is nearly as potent as defiance from solstice for me!
Disvalue via interpersonal expected value and probability
My deontologist friend just told me that treating people like investments is no way to live. The benefits of living by that take are that your commitments are more binding, you actually do factor out uncertainty, because when you treat people like investments you always think “well someday I’ll no longer be creating value for this person and they’ll drop me from their life”. It’s hard to make long term plans, living like that.
I’ve kept friends around out of loyalty to what we shared 5-10 years ago while questioning an expected value theory or probability theory based value prop. So I’m not, like, super guilty of this or anything. But overall I do take expected value theory and probability theory into interpersonal matters, and I don’t object when others do the same for me. Though it’s hard sometimes, I think it’s basically fine if someone drops me because I’m not adding value for them. An edge case in the opposite direction is that you’re obligated to build deep friendships with every acquaintance, which is also a little silly. But a sweet spot, like a marriage or other way of teaming up (like for a project) might meaningfully call for a suspension of expected value theory and probability theory.
Many years ago a mentor told me that critics of abduction point out that induction can make it redundant by making credences in hypotheses about facts, and that this is in fact more aligned with the idea that you don’t have a credence in the facts directly you instead have a credence in some model of the facts. I haven’t spent any time in the literature since then. Overall, do you think abduction is underrated? I do a lot of skimming lesswrong posts about logic and probability and so on and basically never see it.
I may refine this into a formal bounty at some point.
I’m curious if censorship would actually work in the context of blocking deployment of superpowerful AI systems. Sometimes people will mention “matrix multiplication” as a sort of goofy edge case, which isn’t very plausible, but that doesn’t mean there couldn’t be actual political pressure to censor it. A more plausible example would be attention. Say the government threatens soft power against arxiv if they don’t pull attention is all you need, or threatens soft power against harvard if their linguistics department doesn’t pull the pytorch-annotated attention is all you need. By this point, it goes without saying that black hat hackers writing down the equations would face serious consequences if they got caught. Now instead of attention, imagine some more galaxy-brained paper or insight that gets published in 2028 and is an actual missing ingredient to advanced AI (assuming you’re not one of the people who think attention is all you need already is that paper).
While it’s certainly a research project to look at pros and cons of this approach to safety from AI, I think before that we need someone to profile efficacy of technological censorship through history to come at an estimate of how well this would work, i.e., how well it would actually slow or stop the propagation of this information, how well it would slow or stop the deployment of systems based on that information.
My guess at who the ideal person to execute on this bounty would be some patent law nerd, tho I’m sure a variety of types of nerd could do a great job.
I think there exists a generic risk of laundering problem. If you say “capitalism is suboptimal” or “we can do better” people are worried about trojan horses, people worry that you’re just toning it down to garner mainstream support when behind closed doors you’d look more like like “my specific flavor of communism is definitely the solution”. I’m not at all saying I got those vibes from the “transformation of capitalism” post, but that I think it’s plausible someone could get those vibes from it. Notably, the book “Inadequate Equilibria” was explicitly about how capitalism is suboptimal and rigorously asks us if we can improve upon it, and managed not to raise anybody’s alarms about it being a secret communist plot. I guess because it signaled against such a reading by taking the aesthetic/vocabulary of academic econ.
any literature on estimates of social impact of businesses divided by their valuations?
the idea that dollars are a proxy for social impact is neat, but leaves a lot of room for goodhart and I think it’s plausible that they diverge entirely in cases. It would be useful to know, if possible to know, what’s going on here.
Cheers, thanks for writing. I was very anti-math high school student, almost got expelled for throwing a temper tantrum at my algebra 2 teacher cuz I thought it wasn’t fair they were making me sit through it. That was 10th grade and they didn’t make me take any other math courses. 7 or 8 years later I took the placement exam at a community college, placed into precalc I, retreated to khanacademy and retook the exam a few months later placing into calc I, took that and discrete and all of their sequels, ended up getting straight As and tutoring every single math course there. I feel like I owe a lot to khanacademy, a tailored user experience really adds a ton of value over throwing myself at textbooks (and I did eventually figure out how to throw myself at textbooks, but also failed at doing so many times).
The purpose of my comment is to register for anyone intimidated by comments they’ve seen that imply people in the movement were doing math at such n such level in grade school that we’re out here, we exist, and we’re doing stuff; we who had to put effort into precalc in our 20s.
In my (cited by OP) review I say “I think on net it has everything it needs to raise the discourse level”, but something about your comment got me more pessimistic! I’ve been disappointed that I didn’t love it was much as I wanted to since the moment I watched it, but it’s possible I’ll dislike it more over time.
I share OP’s love of Big Short, and I could tell Vice was a regression from that accomplishment. DLU is also a regression from that accomplishment, not just from a filmmaking perspective but from a self-indulgent partisanship perspective.
Methods, famously, includes the line “I am a descendant of the line of Bacon”, tracing empiricism to either Roger (13th century) or Francis (16th century) (unclear which).
Though a cursory wikiing shows an 11th century figure providing precedents for empiricism! Alhazen or Ibn al-Haytham worked mostly optics apparently but had some meta-level writings about the scientific method itself. I found this shockingly excellent quote
The duty of the man who investigates the writings of scientists, if learning the truth is his goal, is to make himself an enemy of all that he reads, and … attack it from every side. He should also suspect himself as he performs his critical examination of it, so that he may avoid falling into either prejudice or leniency.
Should we do more to celebrate Alhazen as an early rationalist?
you’re most likely right about it being harder in the industry!
Are they in charge (of that)? Who chose them?
I don’t think they need permission or an external mandate to do the right thing!
Obviously there are considerations downstream of articulating this, one is that when P(A)>P(B) but V(LA|A)<V(LB|B) so it’s reasonable to hedge on ending up in world B even though it’s not strictly more probable than ending up in world A.