I’m really excited to hear this, and wish you luck :)
My thinking benefited a lot from hanging around CFAR workshops, so for whatever it’s worth I do recommend attending them; my guess is that most people who like reading LessWrong but haven’t tried attending a workshop would come away glad they did.
I’d guess the items linked in the previous comment will suffice? Just buy one mask, two adapters and two filters and screw them together.
I live next to a liberally-polluting oil refinery so have looked into this a decent amount, and unfortunately there do not exist reasonably priced portable sensors for many (I’d guess the large majority) of toxic gasses. I haven’t looked into airplane fumes in particular, but the paper described in the WSJ article lists ~130 gasses of concern, and I expect detecting most such things at relevant thresholds would require large infrared spectroscopy installations or similar.
(I’d also guess that in most cases we don’t actually know the relevant thresholds of concern, beyond those which cause extremely obvious/severe acute effects; for gasses I’ve researched, the literature on sub-lethal toxicity is depressingly scant, I think partly because many gasses are hard/expensive to measure, and also because you can’t easily run ethical RCTs on their effects.
I had the same thought, but hesitated to recommend it because I’ve worn a gas mask before on flights (when visiting my immunocompromised Mom), and many people around me seemed scared by it.
By my lights, half-face respirators look much less scary than full gas masks for some reason, but they generally have a different type of filter connection (“bayonet”) than the NATO-standard 40mm connection for gas cartridges. It looks like there are adapters, though, so perhaps one could make a less scary version this way? (E.g. to use a mask like this with filters like these).
I think the threshold of brainpower where you can start making meaningful progress on the technical problem of AGI alignment is significantly higher than the threshold where you can start making meaningful progress toward AGI.
This is also my guess, but I think required intelligence thresholds (for the individual scientists/inventors involved) are only weak evidence about relative problem difficulty (for society, which seems to me the relevant sort of “difficulty” here).
I’d guess the work of Newton, Maxwell, and Shannon required a higher intelligence threshold-for-making-progress than was required to help invent decent steam engines or rockets, for example, but it nonetheless seems to me that the latter were meaningfully “harder” for society to invent. (Most obviously in the sense that their invention took more person-hours, but I suspect they similarly required more experience of frustration, taking on of personal risk, and other such things which tend to make given populations less likely to solve problems in given calendar-years).
So, I think takeoff has begun, but it’s under quite different conditions than people used to model.
I don’t think they are quite different. Christiano’s argument was largely about the societal impact, i.e. that transformative AI would arrive in an already-pretty-transformed world:
I believe that before we have incredibly powerful AI, we will have AI which is merely very powerful. This won’t be enough to create 100% GDP growth, but it will be enough to lead to (say) 50% GDP growth. I think the likely gap between these events is years rather than months or decades.
In particular, this means that incredibly powerful AI will emerge in a world where crazy stuff is already happening (and probably everyone is already freaking out). If true, I think it’s an important fact about the strategic situation.
I claim the world is clearly not yet pretty-transformed, in this sense. So insofar as you think takeoff has already begun, or expect short (e.g. AI 2027-ish) timelines—I personally expect neither, to be clear—I do think this takeoff is centrally of the sort Christiano would call “fast.”
Makes sense. But on this question too I’m confused—has some evidence in the last 8 years updated you about the old takeoff speed debates? Or are you referring to claims Eliezer made about pre-takeoff rates of progress? From what I recall, the takeoff debates were mostly focused on the rate of progress we’d see given AI much more advanced than anything we have. For example, Paul Christiano operationalized slow takeoff like so:
Given that we have yet to see any such doublings, nor even any discernable impact on world GDP:
… it seems to me that takeoff (in this sense, at least) has not yet started, and hence that we have not yet had much chance to observe evidence that it will be slow?
I think one of the (many) reasons people have historically tended to miscommunicate/talk past each other so much about AI timelines, is that the perceived suddenness of growth rates depends heavily on your choice of time span. (As Eliezer puts it, “Any process is continuous if you zoom in close enough.”)
It sounds to me like you guys (Thane and Ryan) agree about the growth rate of the training process, but are assessing its perceived suddenness/continuousness relative to different time spans?
I have thought of that “Village Idiot and Einstein” claim as the most obvious example of a way that Eliezer and co were super wrong about how AI would go, and they’ve AFAIK totally failed to publicly reckon with it as it’s become increasingly obvious that they were wrong over the last eight years
I’m confused—what evidence do you mean? As I understood it, the point of the village idiot/Einstein post was that the size of the relative differences in intelligence we were familiar with—e.g., between humans, or between humans and other organisms—tells us little about the absolute size possible in principle. Has some recent evidence updated you about that, or did you interpret the post as making a different point?
(To be clear I also feel confused by Eliezer’s tweet, for the same reason).
It seems to me that at least while I worked there (2017-2021), CFAR did try to hash this out properly many times, we just largely failed to converge. I think we had a bunch of employees/workshop staff over the years who were in fact aiming largely or even primarily to raise the sanity waterline, just in various/often-idiosyncratic ways.
we tended in our public statements/fundraisers to try to avoid alienating all those hopes, as opposed to the higher-integrity / more honorable approach of trying to come to a coherent view of which priorities we prioritized how much and trying to help people not have unrealistic hopes in us, and not have inaccurate views of our priorities
I… wish to somewhat-defensively note, fwiw, that I do not believe this well-describes my own attempts to publicly communicate on behalf of CFAR. Speaking on behalf of orgs is difficult, and I make no claim to have fully succeeded at avoiding the cognitive bases/self-serving errors/etc. such things incentivize. But I certainly earnestly tried, to what I think was (even locally) an unusual degree, to avoid such dishonesty.
(I feel broadly skeptical the rest of the org’s communication is well-described in these terms either, including nearly all of yours Anna, but ofc I can speak most strongly about my own mind/behavior).
I basically think you’re being unfair here, so want to challenge you to actually name these or retract.
… So that’s my response to the charge that the param estimates are overly complicated. But I want to respond to one other point you make
It sounds like we’re talking past each other, if you think I’m making two different points. The concern I’m trying to express is that this takeoff model—by which I mean the overall model/argument/forecast presented in the paper, not just the literal code—strikes me as containing confusingly much detail/statistics/elaboration/formality, given (what seems to me like) the extreme sparsity of evidence for its component estimates.
the paper is still very clear and explicit about its limitations
I grant and (genuinely) appreciate that the paper includes many caveats. I think that helps a bunch, and indeed helps on exactly the dimension of my objection. In contrast, I think it probably anti-helped to describe the paper as forecasting “exactly how big” the intelligence explosion will be, in a sense constrained by years of research on the question.
It seems to me that demand for knowledge about how advanced AI will go, and about what we might do to make it go better, currently far outstrips supply. There are a lot of people who would like very much to have less uncertainty about takeoff dynamics, some of whom I expect might even make importantly different decisions as a result.
… and realistically, I think many of those people probably won’t spend hours carefully reading the report, as I did. And I expect the average such person is likely to greatly overestimate the amount of evidence the paper actually contains for its headline takeoff forecast.
Most obviously, from my perspective, I expect most casual readers to assume that a forecast billed as modeling “exactly how big” the intelligence explosion might be, is likely to contain evidence about the magnitude of the explosion! But I see no evidence—not even informal argument—in the paper about the limits that determine this magnitude, and unless I misunderstand your comments it seems you agree?
If AIs that are about as good as humans at broad skills (e.g. software engineering, ML research, computer security, all remote jobs) exist for several years before AIs that are wildly superhuman
I’m curious how likely you think this is, and also whether you have a favorite writeup arguing that it’s plausible? I’d be interested to read it.
I agree the function and parameters themselves are simple, but the process by which you estimate their values is not. Your paper explaining this process and the resulting forecast is 40 pages, and features a Monte Carlo simulation, the Cobb-Douglas model of software progress, the Jones economic growth model (which the paper describes as a “semi-endogenous law of motion for AI software”), and many similarly technical arcana.
To be clear, my worry is less that the model includes too many ad hoc free parameters, such that it seems overfit, than that the level of complexity and seeming-rigor is quite disproportionate to the solidity of its epistemic justification.
For example, the section we discussed above (estimating the “gap from human learning to effective limits”) describes a few ways ideal learning might outperform human learning—e.g., that ideal systems might have more and better data, update more efficiently, benefit from communicating with other super-smart systems, etc. And indeed I agree these seem like some of the ways learning algorithms might be improved.
But I feel confused by the estimates of room for improvement given these factors. For example, the paper suggests better “data quality” could improve learning efficiency by “at least 3x and plausibly 300x.” But why not three thousand, or three million, or any other physically-possible number? Does some consideration described in the paper rule these out, or even give reason to suspect they’re less likely than your estimate?
I feel similarly confused by the estimate of overall room for improvement in learning efficiency. If I understand correctly, the paper suggests this limit—the maximum improvement in learning efficiency a recursively self-improving superintelligence could gain, beyond the efficiency of human brains—is “4-10 OOMs,” which it describes as equivalent to 4-10 “years of AI progress, at the rate of progress seen in recent years.”
Perhaps I’m missing something, and again I’m sorry if so, but after reading the paper carefully twice I don’t see any arguments that justify this choice of range. Why do you expect the limit of learning efficiency for a recursively self-improving superintelligence is 4-10 recent-progress-years above humans?
Most other estimates in the paper seem to me like they were made from a similar epistemic state. For example, half the inputs to the estimate of takeoff slope from automating AI R&D come from asking 5 lab employees to guess; I don’t see any justification for the estimate of diminishing returns to parallel labor, etc. And so I feel worried overall that readers will mistake the formality of the presentation of these estimates as evidence that they meaningfully constrain or provide evidence for the paper’s takeoff forecast.
I realize it is difficult to predict the future, especially in respects so dissimilar from anything that has occurred before. And I think it can be useful to share even crude estimates, when that is all we have, so long as that crudeness is clearly stressed and kept in mind. But from my perspective, this paper—which you describe as evaluating “exactly how dramatic the software intelligence explosion will be”!—really quite under-stresses this.
This model strikes me as far more detailed than its inputs are known, which worries me. Maybe I’m being unfair here, I acknowledge the possibility I’m misunderstanding your methodology or aim—I’m sorry if so!—but I currently feel confused about how almost any of these input parameters were chosen or estimated.
Take your estimate of room for “fundamental improvements in the brain’s learning algorithm,” for example—you grant it’s hard to know, but nonetheless estimate it as around “3-30x.” How was this range chosen? Why not 300x, or 3 million? From what I understand the known physical limits—e.g., Landauer’s bound, the Carnot limit—barely constrain this estimate at all. I’m curious if you disagree, or if not, what constrains your estimate?
For two, a person who has done evil, versus a person who is evil, are quite different things. I think that it’s sadly not always the case that a person’s character is aligned with a particular behavior of theirs.
I do think many of the historical people most widely considered to be evil now were similarly not awful in full generality, or even across most contexts. For example, Eichmann, the ops lead for the Holocaust, was apparently a good husband and father, and generally took care not to violate local norms in his life or work. Yet personally I feel quite comfortable describing him as evil, despite “evil” being a fuzzy folk term of the sort which tends to imperfectly/lossily describe any given referent.
I share the sense that “flaky breakthroughs” are common, but also… I mean, it clearly is possible for people to learn and improve, right? Including by learning things about themselves which lastingly affect their behavior.
Personally, I’ve had many such updates which have had lasting effects—e.g., noticing when reading the Sequences that I’d been accidentally conflating “trying as hard as I can” with “appearing to others to be trying as hard as one might reasonably be expected to” in some cases, and trying thereafter to correct for that.
I do think it’s worth tracking the flaky breakthrough issue—which seems to me most common with breakthroughs primarily about emotional processing, or the experience of quite-new-feeling sorts of mental state, or something like that?—but it also seems worth tracking that people can in fact sometimes improve!
I think the word “technical” is a red herring here. If someone tells me a flood is coming, I don’t much care how much they know about hydrodynamics, even if in principle this knowledge might allow me to model the threat with more confidence. Rather, I care about things like e.g. how sure they are about the direction from which the flood is coming, about the topography of our surroundings, etc. Personally, I expect I’d be much more inclined to make large/confident updates on the basis of information at levels of abstraction like these, than at levels about e.g. hydrodynamics or particle physics or so forth, however much more “technical,” or related-in-principle in some abstract reductionist sense, the latter may be.
I do think there are also many arguments beyond this simple one which clearly justify additional (and more confident) concern. But I try to assess such arguments based on how compelling they are, where “technical precision” is one, but hardly the only factor which might influence this; e.g., another is whether the argument even involves the relevant level of abstraction, or bears on the question at hand.
I think the simple argument “building minds vastly smarter than our own seems dangerous” is in fact pretty compelling, and seems relatively easy to realize beforehand, as e.g. Turing and many others did. Personally, there are not any technical facts about current ML systems which update me more overall either way about our likelihood of survival than this simple argument does.
And I see little reason why they should—technical details of current AI systems strike me as around as relevant to predicting whether future, vastly more intelligent systems will care about us as do e.g. technical details about neuronal firing in beetles about whether a given modern government will care about us. Certainly modern governments wouldn’t exist if neurons hadn’t evolved, and I expect one could in fact probably gather some information relevant to predicting them by studying beetle neurons; maybe even a lot, in principle. It just seems a rather inefficient approach, given how distant the object of study is from the relevant question.
Fwiw, in my experience LLMs lie far more than early Wikipedia or any human I know, and in subtler and harder to detect ways. My spot checks for accuracy have been so dismal/alarming that at this point I basically only use them as search engines to find things humans have said.