Technical staff at Anthropic, previously #3ainstitute; interdisciplinary, interested in everything; ongoing PhD in CS (learning / testing / verification), open sourcerer, more at zhd.dev
Zac Hatfield-Dodds(Zac Hatfield-Dodds)
I agree that expecting a more competent government response than we actually saw (almost anywhere [1]) was entirely reasonable, and that premised on that extreme action to manage tail risks was prudent for the first half of 2020.
Subsequently, I think the rationalist community underperformed wrt. understanding the ongoing crises; we’ve had excellent epistemology but only inconsistently translated that into “winning” as the problem moved from extreme uncertainty and tail risk, to a set of more detail-rich operational challenges. In a slogan, we’ve been long-Sequences and short-CFAR.
[1] In Australia we’ve had a lot of inadequate and needlessly costly policy responses to COVID—for example, I still see more concern about hand hygiene than masks, let alone ventilation—but substantially better than the USA or UK. Between what we did get right, geography, and luck daily life is back to normalish; though the vaccine rollout is inexcusably slow—even with literally nobody dying of COVID (~0.5 daily cases per million people; almost all incoming travellers in supervised quarantine) the economic benefits of moving faster would be huge (allowing potential-superspreader-events again; reduced cost-in-expectation of expensive+unlikely lockdowns). Overall I give Australia as a country a C- on the basis that what we did was barely adequate, and an B+ on the basis that it worked and substantially outperformed most peers.
Seconding Eliezer’s recommendation for the Young Wizards series; I’d call them the most important fiction I’ve read.
And cheap ebooks are available direct from the author, eg https://ebooks.direct/products/the-i-want-everything-youve-got-package (and the Tale of the Five series is probably also of interest to anyone who can spell “polycule”).
It’s worth noticing that this is not a universal property of high-paranoia software development, but a an unfortunate consequence of using the C programming language and of systems programming. In most programming languages and most application domains, crashes only rarely point to security problems.
I disagree. While C is indeed terribly unsafe, it is always the case that a safety-critical system exhibiting behaviour you thought impossible is a serious safety risk—because it means that your understanding of the system is wrong, and that includes the safety properties.
I assume that the learned components (world-model, value function, planner / actor) continue to be updated in deployment—a.k.a. online learning
If it’s not online learning in the strict sense, I’d expect a sufficient (and sufficiently-low-insight) process of updates and redeploys to have the same effect in the medium term. I agree that online learning is the most likely path in such scenarios, but I don’t think it’s necessary.
Fill in the blank: “Think like a ____ about code quality”. What else makes sense?
From my own experience, ‘think like an open-source maintainer’:
One goal is to make it easy (and fun!) enough to work with the code that others will use it and contribute to it - voluntarily, not because they need to for a job or a class. Clarity and brevity are virtues, as is functionality.
The code, in itself, is an instrument for the education of users (and contributors). Readers should be enlightened about the purpose of the code, and at a more granular level focus on “why” rather than “what”. I use “how” comments roughly in proportion to the black-magic-ness of the implementation, for myself as well as others.
Remember that the code is only part of a larger system. Without good docs, runtime UX, community outreach, responses to bug reports and feature requests, etc., the system is less impactful. Code is not an end in itself; only the means to an end. (when thinking like an OSS maintainer; code may of course be an end-in-itself in other ways)
It seems to me that your post is missing something: what specifically do you want people to learn?
I am an educator—currently teaching CS research and Python for a general audience—and I find it’s easy to get people to learn… and surprisingly hard to have everyone learn the same specific thing (to a degree they can use, not just repeat on a test <=2 weeks later). Circumstances to discover and apply the ideas for themselves are best, followed by a variety of communications strategies like the videos, diagrams, and docs you mention.
For code-quality I think you’re asking “how do I help colleagues with different expertise work with this code”, and to me that calls for thinking about communication rather than education. Education includes communication, but it’s larger, slower, harder, and fortunately not necessary here :-)
An alternative framing of this project (for global utilitarians) is “the project of explaining our moral intuitions and other ethical considerations” ala rule utilitiarianism. I don’t see this as a purely empirical project though; there’s also a lot of conceptual work which philosophy is well equipped for. That said,
those inclined to stop at the basic set seem more likely to sit down in the armchair for a bit, answer their central questions with reference to the basic set, then get up and leave — in search of further, empirical information and opportunity relevant to promoting their basic values in practice
rings very true. I enrolled in a double degree in arts/science, including philosophy, but by the end of second year writing another essay exploring how _____ arises from utilitarianism was a lot less enticing than a semester studying {cybernetics, climate, supercomputing, remote sensing, …} and so I graduated with a degree in interdisciplinary sciences instead.
Don’t forget inventions: in the long run, changing the set of available goods and services has been even more important (!) than improving their distribution. Notable post-WWII examples include high-yield cereal varieties, smallpox and polio vaccines, everything made with semiconductors and all the services they enable...
I don’t use Polymarket because, relative to a material investment,
The base rate of crypto institutions losing everything through hacks, fraud, etc. is way too high
On a related note, “USDC” / “Tether” does not inspire confidence
Conversely, volatility—“it would have been a shame if I earned $50,000 betting against Trump but simultaneously lost $500,000 missing out on ETH price changes”
More generally, I haven’t yet seen a prediction market where the “easy money” looks more attractive on a risk-and-work-adjusted basis than working on HypoFuzz. Perhaps others have similar opportunity costs?
With the capital I have on hand as a PhD student, there’s just no way that running something like Vitalik’s pipeline to make money on prediction markets will have a higher excess return-on-hours-worked over holding ETH than my next-best option (which I currently think is a business I’m starting).
If I was starting with a larger capital pool, or equivalently a lower hourly rate, I can see how it would be attractive though.
For a value of “break into flames” that matches damp and poorly-oxygenated fuel, yep! This case in Australia is illustrative; you tend to get a lot of nasty smoke rather than a nice campfire vibe.
You’d have to mismanage a household-scale compost pile very badly before it spontaneously combusts, but it’s a known and common failure mode for commercial-scale operations above a few tons. Specific details about when depend a great deal on the composition of the pile; with nitrate filmstock it was possible with as little as a few grams.
youvegottobekiddingme
The entire project description is full of “we will”, “we aim to”, “we are creating”… without visible evidence that the project has actually made a novel technical thing, I tend to assume that it’s just a cash grab using a pile of buzzwords.
Important for what? Best for what?
In a given (sub)field, the highest-cited papers tend to be those which introduced or substantially improved on a key idea/result/concept; so they’re important in that sense. If you’re looking for the best introduction though that will often be a textbook, and there might be important caveats or limitations in a later and less-cited paper.
I’ve also had a problem where a few highly cited papers propose $approach, many papers apply or puport to extend it, and then eventually someone does a well-powered study checking whether $approach actually works. Either way that’s an important paper, but they tend to be under-cited either because either the results are “obvious” (and usually a small effect) or the field of $approach studies shrinks considerably.
It’s an extremely goodhartable metric but perhaps the best we have for papers; for authors I tend to ask “does this person have good taste in problems (important+tractable), and are their methods appropriate to the task?”.
The hypothesis that Australia succeeded because it was using good epistemics to make decisions is not holding up well in the endgame.
From Australia, this hypothesis was only ever plausible if you looked at high-level outcomes rather than the actual decision-making.
We got basically one thing right: pursue local elimination. Without going into details, this only happened because the Victorian state government unilaterally held their hard lockdown all the way back to nothing-for-two-weeks, ending our winter second wave. Doing so created both a status quo and (having paid higher-than-if-faster costs) a very strong constituency for elimination.
Victoria remains the only area with non-negligible masking. Nationwide, we continue to make expensive and obvious mistakes about handwashing, distancing, quarantine, and appear to be bungling our vaccine rollout.
Zero active cases and zero local transmission covers a multitude of sins. I attribute the result as much to good luck as epistemic skill, and am very glad that COVID is not such a hard problem that we can’t afford mistakes.
- 10 Sep 2021 13:42 UTC; 10 points) 's comment on Covid 9/9: Passing the Peak by (
- 2 Jul 2021 15:09 UTC; 3 points) 's comment on Covid 7/1: Don’t Panic by (
Worldwide demand should be easily big enough to justify [subcontracting manufacturing]
If it was legal to sell vaccines for the market price, or anywhere near their actual value, of course. Thanks to monopsony purchasers (i.e. irrationally cheap governments), we instead see massive underproduction.
On the other hand, there’s some suggestive evidence that seed-stage returns have a power-law distribution with - implying that the best strategy is to filter out the obvious duds and then invest in literally everything else.
Some factors which I think are both important and missing from your model:
Risk. You probably cannot convince me that, in a liquid market, your outperforming trading strategy does not round to “picking up pennies in front of a steamroller”.
Availability of capital. If you have to lock up $10K for a year per 20-hours-of-research deal, you’re probably more constrained by money than time.
Opportunity costs. If you have sufficient quant and business skills to make money trading, you can probably make more working somewhere and investing the proceeds in index funds.
Transaction costs, taxes, etc.
I don’t think we’re actually disagreeing much about outcomes (which I agree have been great!), or even that Australia has competently executed at least enough of the important things to get right. Of the five items you mention I’d include borders, quarantine, snap-lockdowns, and testing as part of the local elimination policy; we haven’t done them perfectly but we have done them well enough.
I understand “using good epistemics to make decisions” to require that your decisions should be made based on a coherent understanding and cost-benefit analysis of the situation, even if both might change over time. “Merely” getting good outcomes doesn’t count!
For example, we still encourage pointless handwashing and distancing while iffy on masks or ventilation—and because we got to zero transmission in other ways, that’s OK. Similarly, it’s true that Australia’s slow vaccine rollout hasn’t cost many lives so far and I hope that neither winter nor variants change that. The cost-in-expectation of an unlikely outbreak should still drive faster vaccination efforts IMO, especially when e.g. increasing local production is not zero-sum.
The simple answer is that micromort exposure is not independent of age: in expectation, a larger proportion of 80-year-old Americans will die within the next day than of 30-year-old Americans.
Based on 1, a 30-year-old is ~6 micromorts/day, rising to ~180 by age 80! On the other hand I’m a little suspicious of their numbers, because the female death rate is lower than male in literally every age group, and by eyeball it seems too much to explain by surviving longer into the 85-and-over group. 2 is a nice overview of life expectancy dynamics, including a section on age-specific mortality.