In general, I think people try to time markets much more than they have skill for. Suppose you think there’s a stock bubble; the temptation is to buy, hold as it rises, and then sell at the top of the bubble before everything comes crashing down. But enough people are trying to do that that you need some special skill in order to be in the leading edge, able to sell when there’s a buyer. It’s much less risky to just sell the stocks as soon as you think there’s a bubble, which foregoes any additional gains but means you avoid the crash entirely (by taking it on voluntarily, sort of).
A similar thing happened at the start of the pandemic, where my plan was basically “look, I don’t think I’m going to be especially good at the risk assessment, I want to just lock down now instead of staying open to get the marginal few weeks of meetings or hangouts or whatever,” and various other friends said “look, I believe in math, I think it’s just paranoia to lock down immediately instead of doing so based on a cost-benefit analysis.” None of us knew it at the time, but the official tests were faulty and delayed and other testing was being suppressed, and so the numbers being used for the cost-benefit analyses were significantly underestimating the true amount of viral transmission.
Also relevant was at the start of the pandemic, there were various border closures and regional quarantines that by design had as little warning as possible. Suppose Disease City has too many cases, and also people in Disease City would want to leave if they think they’re going to get locked in (because regardless of whether or not they have the disease, it’s better for them to be out than in), and the regional / national government wants to lock them in; then the government want to impose the lockdown and then announce it, since that minimizes the number of people who can flee.
Noticeably, if there’s a revolution in some country, it’s much better for the new government to murder all of the people who would want to leave than let them leave. Contrast “Who, after all, speaks today of the annihilation of the Armenians?” with anti-Cuban politics in the US being driven by Cuban expatriates who felt wronged by the government they fled, or various potential leaders kept by other powers as potential puppets ready to be installed when the time is right. If more educated / capitalist Chinese had been allowed to leave China, and ended up in the US, it seems likely could have been a voting bloc much like the Cuban voting bloc, and impacted US-Chinese relations accordingly.
[As it happens, most of the people that I know who speak of the annihilation of the Armenians are themselves part of the Armenian diaspora. Out of the various genocides and purges I’m familiar with, people mostly seem interested in ones to people similar to them, and curiously uninterested in ones that are of their political enemies, or even of the political enemies of people they would like to be allied with.]
In summary, there’s no fire alarm for when to leave the country, in part because the situation is adversarial, and this isn’t the sort of thing you should expect people to be well-calibrated on, since they don’t have many examples to learn from.
I think there’s also a factor where ‘the best time to plant a tree is 20 years ago, and the second best time is today.’ If you expect to want to be in Canada instead of the US in 2030, say, then you might expect to prefer the timeline where you move to Canada in 2020 and live there for ten years to the timeline where you live in the US for nine and a half years and then move to Canada the night before the civil war breaks out (or whatever), and then live there for six months. If the first, you’ll have been able to invest in lots of things that you expect to stick around; in the second, you might skip making useful home improvements because they aren’t mobile. [Similarly, imagining our Chinese intellectuals in the 1950s, if they knew they would want to move to the US in 1965, they might have decided to get the move over with as soon as possible, since then their children might have grown up speaking English, they might have been able to get positions at universities before there was a flood of similar refugees, they might have been able to liquidate Chinese assets at more favorable prices, and so on. Most of the benefit of delaying comes from the hope that actually you won’t have to move; this is why the illusory short-term positive signs are one of the most important parts of the OP.]
This story is less obvious when there’s a huge difference in value between the options. If you’re making net $60k a year in the US and would make net $30k a year in Canada, or whatever, each year you delay moving doubles your effective income. But if the difference is smaller, or you’re weighing different psychological benefits against each other, it’s probably better to get it over with than procrastinate.
Austin checks a lot of the same boxes, except for the hub airport one, and is arguably a better cultural fit. There was some talk in 2018 of Delta making it a “mini-hub”, but who knows where that went. I don’t have enough travel experience in/out of Austin to compare.
I didn’t travel that much out of Austin, and mostly to other hubs, but I never had a bad time and often could get direct flights. The main hassle is just that it’s far from the other places, and so the flights take a while, but that’s always going to be true for at least some people. [I suspect it’s better to be close to some places and further from others than medium distance from everyone, but that’s not obvious.]
Yes, a lot of the increase came from childhood mortality, but life expectancy increased at every age.
Note that life expectancy at 50 and the gap between life expectancy at birth and life expectancy at 1 year basically didn’t budge from 1850 to 1900, whereas life expectancy at birth jumped by 10 years over the same time range. I do think there are at least two distinct things going on (probably all of which are related to increased wealth and improved medical care).
While I agree with you that reducing child mortality is one of the big wins of progress, I have a sense that you’re reasoning about it the wrong way?
Like, I think many (most? Nearly all?) people in the long past did have the view that infants or young children are replaceable, because it was adaptive to their circumstances, and their culture promoted adaptive responses to those circumstances. [Practices like not naming a child for a year are a strategy for the parents to not get too attached, and that makes more sense the less likely it is that a child will last a year.] If they saw the level of attachment our culture encourages parents have to their infants, they would (rightly!) see it as profligate spending only made possible by our massive wealth and technological know-how.
And so in my view, the largest component of the benefit from being in a low infant-mortality world is that parents can afford to treat their children as irreplaceable, which is better for everyone involved. [Like, in the world that’s distant to ameliorate the likely pain of child mortality, also the people who survive have their early experience of the world characterized by distance and low parental investment, including nutritional investment.] The longer you expect things to last, the more you can invest in them—and that goes for relationships and friendships as well.
Ease of joining and leaving.
See also Let Me Go Back, which looks at ‘barrier to exit’ as one of the ‘barriers to entry’; trying something out is expensive not just because you have to switch to it, but also because you have to switch back if you don’t like it.
Could someone elaborate on their thinking?
I think there are lots of different good things that come out of having a primary hub, with pretty different valuations. I basically don’t expect people to have the same goals here, or for them to maybe have them altruistically but not personally.
For example, one big benefit of a hub is that it makes dating easier if you’re looking to date other rationalists (well, except for the whole gender ratio and poly thing). This doesn’t matter to me anymore personally, as I’ve found a long-term partner, but that worked out primarily because I was in the major hub, and so had more options to find a good match. But it still seems like a major benefit of having a primary hub to me; if (say) MIRI wants new hires to be able to date rationalists or EAs, it seems like a good idea to have the office near lots of single rationalists and EAs. [Otherwise, you might find the only people you can hire to work in your volcano lair are the ones that are already married.]
Another benefit of a hub is that you get to have in-person conversations more easily and more spontaneously. I live in a group house that’s organized itself physically to try to maximize ‘organic connection units’, where people end up talking or connecting who otherwise wouldn’t have come into contact. The more connection needs to be scheduled, the less of it you get (because taxes lead to deadweight loss) and also the less serendipity you get (because you only talk to the people who you know about).
I think people are influenced by their surroundings and what people around them are doing; the thing where you know lots of people who care about X is social proof that you too should care about X and effort put into it isn’t wasted. If everyone is living in some random town and just connected to the Craft or Movement through the internet, this increases the chance that they disengage to do something else instead.
A challenge in group preference / info aggregation is distinguishing between “preferences” and “bids.” For example, I might notice that a room is colder than I would like it to be; I might want to share that with the group (so that the decision-making process has this info that would otherwise be mostly private), but I also I’m ignorant of other people’s temperature preferences, and so don’t want to do something unilateralist (where I change the temperature myself) or stake social points on the temperature being changed (in that other people should be dissuaded from sharing their preferences unless they agree, or that others should feel a need to take care of me, or so on).I’ve seen a lot of solutions to this that I don’t like very much, mostly because it feels like this is a short concept but most of the sentences for it are long. (In part because I think a lot of this negotiation happens implicitly, and so making any of it explicit feels like it necessarily involves ramping up the bid strength.)This also comes up in Circling; there’s lots of times when you might want to express curiosity about something without it being interpreted as a demand to meet that curiosity. “I’m interested in this, but I don’t know if the group is interested in this.” I think my current strategy to try is going to ‘jointly naming my individual desire and uncertainty about the aggregate’, but we’ll see how it goes.
attention is zero-sum: there’s a fix supply (well, as a simplification)
As the post notes, zero-sum in resources is not the same as zero-sum in satisfaction. Even if I can only spend a fixed attention budget, how I spend it determine global satisfaction, not just the distribution of satisfaction among players.
Corvee labor is when you pay taxes with your time and effort, instead of your dollars; historically it’s made sense mostly for smaller societies without cash economies. Once you have a cash economy, it doesn’t make very much sense; rather than having everyone spend a month a year building roads, better to have eleven people funding the twelfth person who builds roads, as they can get good at it, and will be the person who is best at building roads, instead of a potentially resentful amateur.America still does this in two ways. The military draft, which was last used in 1973, is still in a state where it could be brought back (the selective service administration still tracks the men who would be drafted if it were reinstated), and a similar program tracks health care workers who could be drafted if needed.
The other is jury duty. Just like one can have professional or volunteer soldiers instead of having conscripted soldiers, one could have professional or volunteer jurors. (See this op-ed that makes the case, or this blog post.) As a result, they would be specialized and understand the law, instead of being potentially resentful amateurs. The primary benefit of a randomly selected jury—that they will (in expectation) represent the distribution of people in society—is lost by the jury selection process, where the lawyers can filter down to a highly non-representative sample. For example, in the OJ Simpson trial, a pool that was 28% black (and presumably 50% female) led to a jury that was 75% black and 83% female. Random selection from certified jurors seems possibly more likely to lead to unbiased juries (tho they will definitely be unrepresentative in some ways, in that they’re legal professionals instead of professionals in whatever randomly selected field).
I’m posting this here because it seems like the sort of thing that is a good idea that is not my comparative advantage to push forward, and nevertheless might be doable with focused effort, and quite plausibly is rather useful, and it seems like a sadder world if people don’t point out the fruit that might be good to pick even if they themselves won’t pick them.
“Completely adversarial” also better captures the strange feature of zero-sum games where doing damage to your opponent, by the nature of it being zero-sum, necessarily means improving your satisfaction, which is a very narrow class of situations.
We could then back out what a rational firm should be willing to invest.
This makes sense, altho I note that I expect the funding here to quite plausibly be ‘irrational.’ For example, some substantial fraction of Microsoft’s value captured is going to global development in a way that seems unlikely to make sense from Microsoft’s bottom line (because Microsoft enriched one of its owners, who then decided to deploy those riches for global development). If building TAI comes out of the ‘altruism’ or ‘exploration’ budget instead of the ‘we expect this to pay back on schedule’ budget, you could see more investment than that last category would justify.
I think NYC was long a ‘second hub’, and there were a bunch of third-tier hubs, but I think the relationships between the hubs never really worked out to make a happy global community. Here’s a post about some previous context. I also suspect that the community has never really had enough people or commitment to have ‘critical mass’ for multiple hubs, and this is part of the problem.
I think there are some systems that have successfully figured this out. I am optimistic about a bunch of current EA student groups at top universities, many of which I visited on the SSC road trip, where there’s both 1) natural recruitment and 2) natural outflow. If someone graduates from Yale and doesn’t stay in New Haven, this is not a surprise; if someone who works as a professional in Austin moves to the Bay Area, this is more of a surprise. This does have a succession problem, where it may be the case that a particular student organizer is great, and once they graduate the group falls apart, but I think at least one university has gone through a few ‘generations’ of organizers, and there’s probably more we can do to support future organizers.
I also think the Mormons have figured this out, where ‘Salt Lake’ controls a bunch of distribution and publishing and so on and is definitely the ‘cultural capital’ of the Mormon world. My sense is that in most places, rather than a weekly sermon cooked up by the local pastor you get a high-production values (in all senses) DVD from the central office. I think our version of that is popular blogs and podcasts, where a global community can be reading SSC and listening to the 80k hours podcast and so mostly be in sync with each other, but this only really works for the “excitement about cool topics” and “gradually fleshing out your world model” dimensions, and is not as good for local community norms or building pairwise relationships or so on.
I think a problem here is that while we have lots of features that are religion-like, I think we don’t really prioritize the “cultural center” aspects, and so there aren’t really people who want to be rationalist pastors / bishops / etc.; Eliezer mostly want to work on AI safety instead of community-building, my sense is that CFAR mostly wants to work on skilling up / recruiting x-risk thinkers instead of community-building, and so on. For example, when I look at plans to make secondary hubs that seem likely to actually happen to me, most of them are parents trying to make good neighborhoods for themselves and their kids, where they are actually taking on the ‘burden of cultural ownership’ or w/e; I think a lot of orgs that people hope will be Community orgs are instead mostly interested in being Craft orgs.
Sure, one could argue that Oakland actually fits the three desiderata, because I left out “low crime,” altho I don’t think Oakland is actually cheap. The broader point of “you get what you pay for” holds, I think, and the only way you get something ‘acceptably cheap’ is deliberately deciding to not pay for some things you could pay for.
My sense is that coordination for this is basically impossible, because of competing access needs. I am most optimistic about versions of this that will:1) Happen even if no one else signs on. [For example, people moving to small towns within commuting distance of SF/Berkeley, where it makes sense for them even if no one else moves to Pinole or Moraga or wherever.]2) Be readily extensible. [If one person buys in Pinole, other people can later buy other houses in Pinole, and slowly shift the balance. Many rationalist group houses started off as a single apartment in a split house, and slowly took over through organic growth. Building a neighborhood of small houses in upstate Vermont to replace your group house, if it works, probably also means someone else could build a subdivision for their group house next door.]3) Pick a vision and be willing to deliver on it. [You’re not going to find a place that has great weather and cheap property value and proximity to great cities; that’s not how efficient markets work. Instead, figure out the few criteria that matter most to you, and do what it takes to achieve those criteria.]This is basically the only way I see for projects to get out of the planning stage and into the reality stage; there will be Some Children Left Behind, and also some people who decide that, well, they do really like the sun but lumenators will be sufficient to make upstate Vermont workable (or whatever).
Separately, I note that ‘chance meetings among extraverts’ seem to be a pretty powerful factor in shaping the history of cultures and organizations, and think that there really is a very large benefit to being in a central hub; I think those hubs have to become much worse for it to not be worth it anymore. [The main compelling reason I see for moving away from the hub is in order to have children—suburbs exist for a reason!--but for people still trying to find partners or meaningful work, the hubs remain very important.]
I think a more attractive option than a group house is something like a pocket neighborhood or baugruppe; I think a location being favorable to that sort of development is a major point in its favor.
To elaborate on this, I think there are two distinct issues: “do they have the right norms?” and “do they do norm enforcement?”. The second is normally good instead of problematic, but makes the first much more important than it would be otherwise. I see Zack_M_Davis as pointing out “hey, if we don’t let people enforce norms because that would make normbreakers feel threatened, do we even have norms?”, which is a valid point, but which feels somewhat irrelevant to the curi question.
For what it’s worth, I think a decision to ban would stand on just his pursuit of conversational norms that reward stamina over correctness, in a way that I think makes LessWrong worse at intellectual progress. I didn’t check out this page, and it didn’t factor into my sense that curi shouldn’t be on LW.
I also find it somewhat worrying that, as I understand it, the page was a combination of “quit”, “evaded”, and “lied”, of which ‘quit’ is not worrying (I consider someone giving up on a conversation with curi understandable instead of shameful), and that getting wrapped up in the “&c.” instead of being the central example seems like it’s defining away my main crux.
Part 1 page 15 talks about “spending on computation”, and assumes spending saturates at 1% of the GDP of the largest country. This seems potentially odd to me; quite possibly the spending will be done by multinational corporations that view themselves as more “global” than “American” or “British” or whatever, and whose fortunes are more tied to the global economy than to the national economy. At most this gives you a factor of 2-3 doublings, but that’s still 4-6 years on a 2-year doubling time.
Overall I’m not sure how much to believe this hypothesis; my mainline prediction is that corporations grow in power and rootlessness compared to nation-states, but it also seems likely that bits of the global economy will fracture / there will be a push to decentralization over centralization, where (say) Alphabet is more like “global excluding China, where Baidu is supreme” than it is “global.” In that world, I think you still see approximately a 4x increase.
I also don’t have a great sense how we should expect the ‘ability to fund large projects’ to compare between the governments of the past and the megacorps of the future; it seems quite plausible to me that Alphabet, without pressure to do welfare spending / fund the military / etc. could put a much larger fraction of its resources towards building TAI, but also presumably this means Alphabet has many fewer resources than the economy as a whole (because there still will be welfare spending and military funding and so on), and on net this probably works out to 1% of total gdp available for megaprojects.
Thanks for sharing this draft! I’m going to try to make lots of different comments as I go along, rather than one huge comment.
[edit: page 10 calls this the “most important thread of further research”; the downside of writing as I go! For posterity’s sake, I’ll leave the comment.]
Pages 8 and 9 of part 1 talk about “effective horizon length”, and make the claim:
Prima facie, I would expect that if we modify an ML problem so that effective horizon length is doubled (i.e, it takes twice as much data on average to reach a certain level of confidence about whether a perturbation to the model improved performance), the total training data required to train a model would also double. That is, I would expect training data requirements to scale linearly with effective horizon length as I have defined it.
I’m curious where ‘linearly’ came from; my sense is that “effective horizon length” is the equivalent of “optimal batch size”, which I would have expected to be a weirder function of training data size than ‘linear’. I don’t have a great handle on the ML theory here, tho, and it might be substantially different between classification (where I can make batch-of-the-envelope estimates for this sort of thing) and RL (where it feels like it’s a component of a much trickier system with harder-to-predict connections).
Quite possibly you talked with some ML experts and their sense was “linearly”, and it makes sense to roll with that; it also seems quite possible that the thing to do here is have uncertainty over functional forms. That is, maybe the effective horizon scales linearly, or maybe it scales exponentially, or maybe it scales logarithmically, or inverse square root, or whatever. This would help double-check that the assumption of linearity isn’t doing significant work, and if it is, point to a potentially promising avenue of theoretical ML research.
[As a broader point, I think this ‘functional form uncertainty’ is a big deal for my timelines estimates. A lot of people (rightfully!) dismissed the standard RL algorithms of 5 years ago for making AGI because of exponential training data requirements, but my sense is that further algorithmic improvement is mostly not “it’s 10% faster” but “the base of the exponent is smaller” or “it’s no longer exponential.”, which might change whether or not it makes sense to dismiss it.]
A simple, well-funded example is autonomous vehicles, which have spent considerably more than the training budget of AlphaStar, and are not there yet.
I am aware of other examples that do seem to be happening, but I’m not sure what the cutoff for ‘massive’ should be. For example, a ‘call center bot’ is moderately valuable (while not nearly as transformative as autonomous vehicles), and I believe there are many different companies attempting to do something like that, altho I don’t know how their total ML expenditure compared to AlphaStar’s. (The company I’m most familiar with in this space, Apprente, got acquired by McDonalds last year, who I presume is mostly interested in the ability to automate drive-thru orders.)
Another example that seems relevant to me is robotic hands (plus image classification) at sufficient level that warehouse pickers could be replaced by robots.