DirectedEvolution

For Bet­ter Com­ment­ing, Avoid PONDS

Crit­i­cism of some pop­u­lar LW articles

(Try­ing To) Study Text­books Effec­tively: A Year of Experimentation

• DirectedEvolution 24 Apr 2023 4:25 UTC

  64 points

  33

  in reply to: habryka’s comment on: Con­tra Yud­kowsky on AI Doom

  I feel like even under the worldview that your beliefs imply, a superintelligence will just make a brain the size of a factory, and then be in a position to outcompete or destroy humanity quite easily.

  I’m going to expand on this.

  Jacob’s conclusion to the speed section of his post on brain efficiency is this:

  The brain is a million times slower than digital computers, but its slow speed is probably efficient for its given energy budget, as it allows for a full utilization of an enormous memory capacity and memory bandwidth. As a consequence of being very slow, brains are enormously circuit cycle efficient. Thus even some hypothetical superintelligence, running on non-exotic hardware, will not be able to think much faster than an artificial brain running on equivalent hardware at the same clock rate.

  Let’s accept all Jacob’s analysis about the tradeoffs of clock speed, memory capacity and bandwidth.

  The force of his conclusion depends on the superintelligence “running on equivalent hardware.” Obviously, core to Eliezer’s superintelligence argument, and habryka’s comment here, is the point that the hardware underpinning AI can be made large and expanded upon in a way that is not possible for human brains.

  Jacob knows this, and addresses it in comments in response to Vaniver pointing out that birds may be more efficient than jet planes in terms of calories/​mile flown, but that when the relevant metric is top speed or human passengers carried, the jet wins. Jacob responds:

  I agree electricity is cheap, and discuss that. But electricity is not free, and still becomes a constraint...

  The rental price using enterprise GPUs is at least 4x as much, so more like $20,000/​yr per agent. So the potential economic advantage is not yet multiple OOM. It’s actually more like little to no advantage for low-end robotic labor, or perhaps 1 OOM advantage for programmers/​researchers/​ec. But if we had AGI today GPU prices would just skyrocket to arbitrage that advantage, at least until foundries could ramp up GPU production.

  So the crux here appears to be about the practicality of replacing human brains with factory-sized artificial ones, in terms of physical resource limitations.

  Daniel Kokotajlo disagrees that this is important:

  $2,000/​yr per agent is nothing, when we are talking about hypothetical AGI. This seems to be evidence against your claim that energy is a taut constraint.

  Jacob doubles down that it is:

  Energy is always an engineering constraint: it’s a primary constraint on Moore’s Law, and thus also a primary limiter on a fast takeoff with GPUs (because world power supply isn’t enough to support net ANN compute much larger than current brain population net compute).

  But again I already indicated it’s probably not a ‘taut constraint’ on early AGI in terms of economic cost—at least in my model of likely requirements for early not-smarter-than-human AGI.

  Also yes additionally longer term we can expect energy to become a larger fraction of economic cost—through some combination of more efficient chip production, or just the slowing of moore’s law itself (which implies chips holding value for much longer, thus reducing the dominant hardware depreciation component of rental costs)

  So Jacob here admits that energy is neither a ‘taut constraint’ for early AGI, and that at the same time it will be a larger fraction of the cost. In other words, it’s not a bottleneck for AGI, and no other resource is either.

  This is where Jacob’s discussion ended.

  So I think Jacob has at least two jobs to do to convince me. I would be very pleased and appreciative if he achieved just one of them.

  First, he needs to explain why any efficiency constraints can’t be overcome by just throwing a lot of material and energy resources into building and powering inefficient or as-efficient-as-human-brains GPUs. If energy is not a taut constraint for AGI, and it’s also expected to be an increasing fraction of costs over time, then that sounds like an argument that we can overcome any efficiency limits with increasing expenditures to achieve superhuman performance.

  Second, he needs to explain why things like energy, size, or ops/​sec efficiency are the most important efficiency metrics as opposed to things like “physical tasks/​second,” or “brain-size intelligences produced per year,” or “speed at which information can be taken in and processed via sensors positioned around the globe.” There are so very many efficiency (“useful output/​resource input”) metrics that we can construct, and on many of them, the human brain and body are demonstrably nowhere near the physical limit.

  Right now, doubling down on physics-based efficiency arguments, as he’s doing here, don’t feel like a winning strategy to me.

Here’s Why I’m He­si­tant To Re­spond In More Depth

Us­ing a mem­ory palace to mem­o­rize a text­book.

Sum­mary of a new study on out-group hate (and how to fix it)

• DirectedEvolution 24 Apr 2023 3:07 UTC

  60 points

  45

  in reply to: ShardPhoenix’s comment on: Con­tra Yud­kowsky on AI Doom

  I had the same impression at first, but in the areas where I most wanted these, I realized that Jacob linked to additional posts where he has defended specific claims at length.

  Here is one example:

  EY is just completely out of his depth here: he doesn’t seem to understand how the Landauer limit actually works, doesn’t seem to understand that synapses are analog MACs which minimally require OOMs more energy than simple binary switches, doesn’t seem to understand that interconnect dominates energy usage regardless, etc.

  I usually find Tyler Cowenesque (and heck, Yudkowskian) phrases like this irritating, and usually they’re pretty hard to interrogate, but Jacob helpfully links to an entire factpost he wrote on this specific point, elaborating on this claim in detail.
  

  He does something similar here:

  EY derived much of his negative beliefs about the human mind from the cognitive biases and ev psych literature, and especially Tooby and Cosmide’s influential evolved modularity hypothesis. The primary competitor to evolved modularity was/​is the universal learning hypothesis and associated scaling hypothesis, and there was already sufficient evidence to rule out evolved modularity back in 2015 or earlier.

  He is doing an admirable job of summarizing the core arguments of Eliezer’s overall model of AI risk, then providing deeply-thought-out counterarguments. The underlying posts appear to be pretty well-cited.

  That doesn’t mean that every claim is backed by an additional entire post of its own, or that every argument is convincingly correct.

  for various reasons mostly stemming from the strong prior that biology is super effecient I expect humanity to be very difficult to kill in this way (and growing harder to kill every year as we advance prosaic AI tech)

  This is more of the Cowenesque/​Yudkowskianesque handwavey appeal to authority, intuition, and the implication that any informed listener ought to have mastered the same background knowledge and come naturally to the same conclusions. Here, my response would be “viruses are not optimizing to kill humans—they are optimizing for replication, which often means keeping hosts alive.”

  He follows this with another easy-to-argue-with claim:

  killing humanity would likely not be in the best interests of even unaligned AGI, because humans will probably continue to be key components of the economy (as highly efficient general purpose robots) long after AGI running in datacenters takes most higher paying intellectual jobs

  This sounds at best like an S-risk (how would the AI cause humans to behave like highly efficient general purpose robots? Probably not in ways we would enjoy from our perspective now). And we don’t need to posit that an unaligned AI would be bent on keeping any semblance of an “economy” running. ChaosGPT 5.0 might specifically have the goal of destroying the entire world as efficiently as possible.

  But my goal in articulating these counterarguments isn’t because I’m hoping to embroil myself in a debate downthread. It’s to point out that while Jacob’s still very far from dealing with every facet of Eliezer’s argument, he is at the same time doing what I regard as a pretty admirable job of interrogating certain specific claims in depth.

  And Jacob doesn’t read to me as making what I would call “personal attacks on Eliezer.” He does do things like:

  Accurately paraphrase arrogant-sounding things Eliezer says

  Moreover he predicts that this is the default outcome, and AI alignment is so incredibly difficult that even he failed to solve it.

  Accurately describe the confidence with which Eliezer makes his arguments (“brazenly”).

  Be very clear and honest about just how weak he finds Eliezer’s argument, after putting in a very substantial amount of work in order to come to this evaluation

  I have evaluated this model in detail and found it substantially incorrect and in fact brazenly naively overconfident.

  Every one of his key assumptions is mostly wrong, as I and others predicted well in advance. EY seems to have been systematically overconfident as an early futurist, and then perhaps updated later to avoid specific predictions, but without much updating his mental models (specifically his nanotech-woo model, as we will see).

  Offer general reasons to think the epistemics are stronger on his side of the debate:

  Even though the central prediction of the doom model is necessarily un-observable for anthropic reasons, alternative models (such as my own, or moravec’s, or hanson’s) have already made substantially better predictions, such that EY’s doom model has low posterior probability.

  EY has espoused this doom model for over a decade, and hasn’t updated it much from what I can tell.

  Note: this is one area I do think Jacob’s argument is weak—I would need to see one or more specific predictions from Jacob or the others he cites from some time ago in order to evaluate this claim. But I trust it is true in his own mind and could be made legible.

  Overall, I think it is straightforwaredly false to say that this post contains any “personal attacks on Eliezer.”

  Edit: However, I also would say that Jacob is every bit as brazen and (over?)confident as Eliezer, and has earned the same level of vociferous pushback where his arguments are weak as he has demonstrated in his arguments here.

  And I want to emphasizes that in my opinion, there’s nothing wrong with a full-throated debate on an important issue!

The Multi-Tower Study Strategy

The Good Try Rule

• DirectedEvolution 8 Feb 2023 2:03 UTC

  55 points

  65

  on: Not­ing an er­ror in Inad­e­quate Equilibria

  Just to say it explicitly, I also don’t see at all how the graphs you’ve shown could be interpreted to support Eliezer’s interpretations. I also don’t understand how Eliezer arrived at the decision to tell this story in this way based on this data.

  My only background on this issue is reading Inadequate Equilibria and this post, so that’s all I’m going on here.

Aging and the gero­science hypothesis

• DirectedEvolution 19 Oct 2021 6:19 UTC

  54 points

  on: Lies, Damn Lies, and Fabri­cated Options

  An alternative and maybe more fun term for this is a “comb-over.”

  If you’re losing your hair, you might think that one of your options is to disguise the fact that you’re balding by giving yourself a comb-over. But in fact, that’s a fabricated option. Nobody will be fooled, and you’ll look worse than you would if you left your hair alone or shaved it all off.

  So we could say that a price gouging ban is a “comb-over for supply constraints.” An abortion ban is a “comb-over for dangerous or unwanted pregnancies.” Staying with an unstable partner is giving their issues a comb-over. Banning block lists is giving a comb-over to social discord. Forcing your kid to do things they don’t like is giving a comb-over to their independence.

  I like this term because it’s a little funny and it gets the point across in a way that I might actually use in a conversation with a friend. If anybody else likes this, I give Duncan the credit, because I wouldn’t have come up with this if Duncan hadn’t focused attention on it.

• DirectedEvolution 6 Jul 2021 5:10 UTC

  54 points

  0

  on: The topic is not the content

  Looking back on the ten years I spent teaching piano lessons to children, do I think that this framework would have helped me decide if it was a good career?

  The topic was music. One way of describing the content was a combination of sitting down with children, managing behavior, explaining how to read music, correcting errors, negotiating with parents, accounting, driving between houses, buying music books, workshopping learning and emotional challenges, inventing new approaches to teaching songwriting and improvisation, and providing encouragement. I could have anticipated that with forethought.

  That dry verbal description captures absolutely nothing of the rich and rewarding experience I had teaching music lessons. And just getting a sense of the day-to-day, as by job shadowing another teacher, would have also been more deceptive than descriptive. Having relationships with children that evolve and deepen over the course of years, experimenting with solutions to problems and having the satisfaction of seeing some of those solutions work out, and creating musical performances at nursing homes that put a smile on the face of the elderly residents, are experiences that become vastly more interesting and meaningful when it’s your life, rather than somebody else’s.

  So my concern with this approach of focusing on the content, rather than the topic, is that I suspect I’d have been really bad at anticipating the content. And the more work I put into it, I suspect the more confident that I would have become in my misinterpretation. Coming into a job with a sense of mission and purpose, but an open-ended sense of what it was actually going to be like, actually seems to me like a better approach than trying to somehow imagine how the day-to-day experience will feel like from the inside, and then trying to choose from the imagined option that seems the most attractive a priori.

  For me, the focus on the topic of music, rather than the content of teaching, helped spark my imagination for what music lessons could be like. I wanted to do something very different from the way I’d been taught growing up.

  Maybe another way of pointing at the same thing is that topic/​content is not actually the division that makes the most sense to me. Instead, it’s something like “psychology/​behavior.” It’s the division between what the job feels like to do, in terms of the emotions, ideas, and stories involved, versus the concrete physical and social activities you perform on a daily basis. The former seems much more important to me than the latter, from a meaningfulness perspective. Behavior is important for your experience, in terms of energy expenditure, risks of injury, and so on. But meaningfulness is often a big component of what keeps people at their jobs, and I just don’t think that a focus on “content” would have helped me connect with or anticipate that aspect of my previous line of work.

  What links here?

  • Ruby's comment on The topic is not the content by Aaron Bergman (17 Jul 2021 16:08 UTC; 6 points)

• DirectedEvolution 14 Feb 2021 7:04 UTC

  54 points

  in reply to: [DEACTIVATED] Duncan Sabien’s comment on: “PR” is cor­ro­sive; “rep­u­ta­tion” is not.

  PR is about managing how an antagonist could distort your words and actions to portray you in a negative light.

  By contrast, for the concept of “honor” to mean anything, you have to be imagining that there’s a community of people who care about honor and approach that question with integrity. It assumes a level of charity and sophistication in the people you’re appealing to.

Let’s Re­name Our­selves The “Me­tacog­ni­tive Move­ment”

• DirectedEvolution 2 Feb 2023 4:43 UTC

  51 points

  5

  in reply to: jasoncrawford’s comment on: Can we “cure” can­cer?

  The germ theory of disease is, most essentially, the theory that infectious diseases are caused by invasion of the patient’s body by a pathogen. It is defined by the type of thing that is the root cause of the disease—in this case, a non-human cell, virus, or even a malformed protein.

  The direct equivalent in cancer is the theory that the cancer is made from a human’s own cells growing out of control. That’s a universally accepted fact and it has been for a long time. Again, I know that you know this, so I’m just really unclear about why you’re proposing that we don’t have the equivalent of germ theory for cancer.

  But let’s elaborate, because I am in biomedical engineering and it might be that I’m acquiring a little bit of expert syndrome, assuming that facts that are obvious to anyone in the field are equally obvious to those who are not.

  You are correct in saying that the difficulty with cancer is fundamentally one of targeting! The nastiest cancer cells have evolved a range of capabilities:

  • They look like healthy cells, so the immune system doesn’t want to destroy them

  • They produce signaling proteins that tell the immune system to calm down, which we call an “antiinflammatory microenvironment”

  • They evolve resistance to chemotherapies, so that a drug and dose that would kill the cancer will also prove so lethal to healthy human cells that there’s no benefit to administering it anymore

  • They spread to new locations throughout the body. Since drugs don’t always work equally well in all tissues, they might find a safe haven when they metastasize

  • They rearchitect tissue in ways that are compatible with their own growth and survival

  • They weaken the patient’s physical health and physically infiltrate crucial tissue, making it riskier to operate

  Figuring out a way to either get a drug to the physical location of the cancer cells but not to the healthy cells, or finding a drug that will selectively damage cancer cells but not healthy cells, is a huge challenge. Since cancer cells are not only alive and capable of evolution, but thrive off the same bodily conditions that normal cells survive best on, it’s much harder to destroy them than it is to destroy bacteria (a challenge which is also becoming harder with the evolution of antibiotic resistance).

  Fortunately, we are in the midst of figuring out a range of solutions to overcome these targeting challenges. Here are some examples, several of which we’re actively doing in my own lab:

  • Cancer creates an acidic pH nearby, and we can devise drug delivery systems that are activated by low pH

  • We can take out patient immune cells, equip them with genetically engineered detectors for the patient’s specific cancer, and then re-introduce them into the host

  • We can drive a pro-inflammatory state at the cancer site so that the host immune cells can eliminate the cancer instead of getting shut down when they’re nearby

  • We can destroy all the cancer cells with really severe treatments that would normally kill the patient, but save and reintroduce the cells needed to allow the host to regenerate and survive after the treatment

  • Some cancers are caused by infectious diseases, and we can vaccinate against those

  • We can do genetic testing for things like breast cancer-promoting genes, and women can choose to have a preventative masectomy to reduce their risk from these genes. That’s just one well-known example, other non-vital organs can be removed from both men and women as needed if they are at risk of developing cancer.

  • We can carefully break down the architecture of the cancer to figure out how it grows and develops, then carefully interrupt crucial links in that process in ways specific to individual cancers

  • We can identify and eliminate mutation-causing carcinogens from our environment

  • We can engineer synthetic or cell-based replacement organs, so that if excising the cancer would normally require destroying a critical organ, we can just take it out and then replace it with a new organ that doesn’t have any cancer cells in it

  • We can create sensors for cancer that detect it at an earlier stage when it’s easier to treat

  • We can look for things besides just cell division that the cancer needs to do more than normal, like growing new blood vessels, and inhibit those activities

  • We have an ever-expanding list of ways to target cell growth and division, which cancers do more than normal human cells (but which human cells also do, which is why you lose your fast-growing hair during chemo).

  This is by no means an exhaustive list, cancer isn’t even my research topic, this is just stuff I’ve picked up from my labmates who work directly on cancer research.

  The reason we have so many different strategies for tackling cancer, all tractable, is that we understand cancer really in quite a lot of detail, such that even though it’s a really difficult collection of illnesses to treat due to being made of our own cells running amok, we are still able to deal with many cancers rather successfully.

  There are other diseases of old age for which I’d be much more comfortable with the statement that we have no established, consensus basic understanding of how the disease works. Alzheimer’s is one. Aging itself is by no means pre-theoretical, we have many decades of evolutionary theory to explain it in principle and nowadays a lot of specific biological mechanisms to flesh out the picture, so to speak.

  Cancer is a case where the reason we haven’t cured it, despite decades of intense research, is that it’s a really tough problem and we’ve only developed the kinds of sophisticated tools that might let us bring the hammer down on it in the last couple decades. As you know, FDA trials take a long time, and so it takes a long time to move from basic research to the clinic.

  You can see our progress though at the CDC, which shows a steadily decreasing death rate from cancer over the last quarter century.

  

  And remember that not all of the deaths that do occur are failures of medicine. Some are because patients do the unhealthy stuff we know gives them cancer anyway, despite warnings, or they’re not proactive about checkups, or they don’t have access to care, or they refuse or can’t afford treatment.

  Meanwhile, we are continuing to build up our technology stack, finding ways to do things like safer and more robust gene therapy. Remember that CRISPR for all its hype is like a decade old discovery, so drugs that exploit the refinements we’ve made to it since then (which are considerable) have really only had a few years maybe to get tested in preclinical models.

  Whatever medications are coming out today are the biomedical equivalent of ancient light from distant stars, showing us the picture of what biomedicine was capable of 15 years ago. In a regime of explosive exponential growth in technology, particularly biotech, there’s going to be a huge gap between the insane stuff we can engineer in the lab today and what patients are able to take advantage of in the hospital. That might look like some sort of stagnation, but it’s not, it’s just the nature of the beast when human lives are on the line if you fuck up the technology and the rules aren’t built from the ground up with math the way they are on computers. Personally, I’m thrilled with the biomedical universe and it feels like it’s moving insanely fast, but I understand why it might not look that way if you’re not working in the field as a professional because of the time delay between discovery and drug.

  What links here?

  • Can we “cure” can­cer? by jasoncrawford (1 Feb 2023 22:03 UTC; 41 points)
  • Progress links and tweets, 2023-02-08 by jasoncrawford (8 Feb 2023 15:52 UTC; 10 points)
  • Ja­son’s links and tweets, 2023-02-08 by jasoncrawford (Progress Forum; 8 Feb 2023 15:52 UTC; 5 points)
  • Can we “cure” can­cer? by jasoncrawford (Progress Forum; 1 Feb 2023 22:03 UTC; 4 points)

Pro­gres­sive High­light­ing: Pick­ing What To Make Into Flashcards

• DirectedEvolution 14 Jun 2021 4:38 UTC

  49 points

  0

  on: Precognition

  It’s almost impossible to predict the future. But it’s also unnecessary, because most people are living in the past. All you have to do is see the present before everyone else does.

  That really is a nice quote, though.

An alarm bell for the next pandemic