I think I mostly disagree—any decent WBE project would not and should not jump to directly emulating the human mind. That seems dumb and not something I’d do if I was in charge. It seems obvious to me that you start with smaller non-AGI brains like fruit-flies, then mice, then maybe small primates first. I’m not saying this would completely eliminate all the other issues.
But brains have evolved (and thus similar to each other). Things like whether or not protein complexes and brain flow is important can be found out much much earlier that we ever get close to putting a general intelligence capable of self-modifying on a computer.
I’d like to at some point write-up why I now think WBE is probably a much better way to get to safe world than other things, but I think something like this in order of importance:
- WBE is much more easier to see progress in than AI / human intelligence enhancement.
(A) You have faster feedback loops with a scanned brain, than something like bio-humans where you have to wait a few years to administer your first tests, and potentially much longer to see long term effects.
(B) You can actually tell if you’re making progress as opposed to AI alignment. Non-genius existing humans can administer and verify a battery of tests to ensure e.g your mouse brain emulation is high-fidelity.
Even if you do human enhancement first, you’d probably want them to pursue something like a high-fidelity WBE first rather than setting up a AGI. Regular non-enhanced humans wouldn’t be able to tell if what they’re doing is safe anyway. Or you have to wait long enough for all humans to be enhanced over maybe multiple generations or more.
With a very high probability, I’d claim that an emulated mouse brain would not lead to runaway ASI no matter how long it’s run. And I think that’s mostly true even upto chimp level given we know the speeds they’re running at. If you can emulate a chimp (or even a lower level primate at high fidelity), then you can probably have a safe human WBE without understanding anything more (again claimed with no evidence). Maybe you’ve unlocked some powerful learning algorithms while emulating a monkey, and. this leads to the AICo co-opting this idea. Which brings me to my next point:
In a realistic world, I think a big problem you bring up—better emulations somehow leaking into AI co’s capabilities is mostly moot. The AI companies are having zero problems producing capability increases anyway. Either we ban AI research completely, and then this point is moot, or we don’t ban it, and we get the AI labs creating superintilligence anyway. I think this is more of a what happens next? Once we ban AGI research, I think subsidizing / funding WBE is still the safest thing to do.
I’d claim that an emulated mouse brain would not lead to runaway ASI no matter how long it’s run
This is not my claim. My claim is more like this:
While you were scaling to a human brain, someone else could take the mouse brain, or reconstruct it using similar methods. Then they could multiply the mouse cortex by and use machine learning (since, unlike you, they are crazy and reckless) to train their weird hypermouse to be highly performant at many tasks; they can try whatever interconnectivity hyperparameters they want, biorealistic or not, and they can try any neural learning algorithms they want, because they’re crazy and don’t care about AI capabilities risk. They seem to have a lot of tools for expropriating this stuff, which you did help to make available.
You might think scaling a mouse brain like that can’t feasibly lead to major increases in AI capabilities. I disagree, but ok. So next you do a small primate. Are you going to also claim that the small primate, scaled up by , can’t be ML trained to be superhumanly intelligent? Ok. What about the chimp you do as a test? That can’t be made superhuman with scaling up and training? This kinda beggars belief, though maybe I’m missing something?
In a realistic world, I think a big problem you bring up—better emulations somehow leaking into AI co’s capabilities is mostly moot. The AI companies are having zero problems producing capability increases anyway. Either we ban AI research completely, and then this point is moot, or we don’t ban it, and we get the AI labs creating superintilligence anyway. I think this is more of a what happens next?
I don’t think it’s moot. Part of why it may be very difficult to pause for a very long time is that new powerful AI algorithms will keep being discovered because it’s hard to track theoretical / conceptual / small-compute research. But maybe we can use strong social norms to prevent people from trying to do conceptual AGI research. Ok, but then this WBE research is a kind of covert AGI algorithms research, exactly the sort of thing that erodes the pause despite best efforts.
I don’t think the AI companies have “zero problems” producing capabilities. I mean, that kinda contradicts the whole WBE plan, doesn’t it? Like, the WBE plan, at least in my head, is:
Create a WBE.
The WBE self-modifies themself to increase intelligence.
Using their increased intelligence, they safely prevent unaligned AGIs from being created.
This plan assumes that there’s strategically decisive headroom in intelligence, available to a WBE, but not yet acquired by the AIs at AI companies. So, even the initial WBE has abilities not yet acquired by the AIs (namely, the ability to quickly increase their own intelligence through self-modification). So if you do a bunch of research and create a WBE, you have at some point figured out how to put that novel capability, not held by AI companies, into a computer.
Once we ban AGI research, I think subsidizing / funding WBE is still the safest thing to do.
| Are you going to also claim that the small primate, scaled up by , can’t be ML trained to be superhumanly intelligent? Ok. What about the chimp you do as a test? That can’t be made superhuman with scaling up and training? This kinda beggars belief, though maybe I’m missing something?
My claim is more that if we do get a chimp running at 10^3 and it somehow turns into ASI, I believe the resulting ASI is likely to be much more aligned, and we probably all get to live rather whatever we get scaling GPT6 etc.
I think that timelines for WBE if we’re trying to compete with the current AI companies in the free-market are substantially unlikely to happen (<5%). There’s a lot of tech missing for mass-manufacturing scanners etc, and to your point, I’m not sure the mouse-AI leads to immediate returns / revenue that you can go and get a bunch of VC funding for it anyway. Not enough to get to a human WBE fully funded by existing market mechanisms.
| I don’t think the AI companies have “zero problems” producing capabilities. I mean, that kinda contradicts the whole WBE plan, doesn’t it?
I think a human brain would have decisive headroom compared to current AIs. Potentially whatever is needed for RSI doesn’t exist in current AI or maybe never under the current regime, but it does exist in the human mind.
I also feel that a neuron-level scanning and emulation would not lead to any direct intelligence-algorithms breakthroughs. I think I could buy this argument pre LLMs. But we have current AIs where already know every single activation, and we don’t have any level of algorithmic understanding of what’s going on inside that leads us closer to any theory of intelligence.
My claim is more that if we do get a chimp running at 10^3 and it somehow turns into ASI, I believe the resulting ASI is likely to be much more aligned, and we probably all get to live rather whatever we get scaling GPT6 etc.
Ok but I don’t think this is very relevant? My point is that, before you get something powerful & aligned, before that point, you got something that someone else could turn into something powerful & not-aligned by doing non-safe machine learning with your thing as a component / ingredient.
I think a human brain would have decisive headroom compared to current AIs. Potentially whatever is needed for RSI doesn’t exist in current AI or maybe never under the current regime, but it does exist in the human mind.
So then if a project gets a true full human WBE, before that point it created capabilities, not already present in an AI, but yes in a computer. At that point, those capabilities could be expropriated and trained on by AI capabilities researchers.
I think I mostly disagree—any decent WBE project would not and should not jump to directly emulating the human mind. That seems dumb and not something I’d do if I was in charge. It seems obvious to me that you start with smaller non-AGI brains like fruit-flies, then mice, then maybe small primates first. I’m not saying this would completely eliminate all the other issues.
But brains have evolved (and thus similar to each other). Things like whether or not protein complexes and brain flow is important can be found out much much earlier that we ever get close to putting a general intelligence capable of self-modifying on a computer.
I’d like to at some point write-up why I now think WBE is probably a much better way to get to safe world than other things, but I think something like this in order of importance:
- WBE is much more easier to see progress in than AI / human intelligence enhancement.
(A) You have faster feedback loops with a scanned brain, than something like bio-humans where you have to wait a few years to administer your first tests, and potentially much longer to see long term effects.
(B) You can actually tell if you’re making progress as opposed to AI alignment. Non-genius existing humans can administer and verify a battery of tests to ensure e.g your mouse brain emulation is high-fidelity.
Even if you do human enhancement first, you’d probably want them to pursue something like a high-fidelity WBE first rather than setting up a AGI. Regular non-enhanced humans wouldn’t be able to tell if what they’re doing is safe anyway. Or you have to wait long enough for all humans to be enhanced over maybe multiple generations or more.
With a very high probability, I’d claim that an emulated mouse brain would not lead to runaway ASI no matter how long it’s run. And I think that’s mostly true even upto chimp level given we know the speeds they’re running at. If you can emulate a chimp (or even a lower level primate at high fidelity), then you can probably have a safe human WBE without understanding anything more (again claimed with no evidence). Maybe you’ve unlocked some powerful learning algorithms while emulating a monkey, and. this leads to the AICo co-opting this idea. Which brings me to my next point:
In a realistic world, I think a big problem you bring up—better emulations somehow leaking into AI co’s capabilities is mostly moot. The AI companies are having zero problems producing capability increases anyway. Either we ban AI research completely, and then this point is moot, or we don’t ban it, and we get the AI labs creating superintilligence anyway. I think this is more of a what happens next? Once we ban AGI research, I think subsidizing / funding WBE is still the safest thing to do.
This is not my claim. My claim is more like this:
While you were scaling to a human brain, someone else could take the mouse brain, or reconstruct it using similar methods. Then they could multiply the mouse cortex by
You might think scaling a mouse brain like that can’t feasibly lead to major increases in AI capabilities. I disagree, but ok. So next you do a small primate. Are you going to also claim that the small primate, scaled up by
I don’t think it’s moot. Part of why it may be very difficult to pause for a very long time is that new powerful AI algorithms will keep being discovered because it’s hard to track theoretical / conceptual / small-compute research. But maybe we can use strong social norms to prevent people from trying to do conceptual AGI research. Ok, but then this WBE research is a kind of covert AGI algorithms research, exactly the sort of thing that erodes the pause despite best efforts.
I don’t think the AI companies have “zero problems” producing capabilities. I mean, that kinda contradicts the whole WBE plan, doesn’t it? Like, the WBE plan, at least in my head, is:
Create a WBE.
The WBE self-modifies themself to increase intelligence.
Using their increased intelligence, they safely prevent unaligned AGIs from being created.
This plan assumes that there’s strategically decisive headroom in intelligence, available to a WBE, but not yet acquired by the AIs at AI companies. So, even the initial WBE has abilities not yet acquired by the AIs (namely, the ability to quickly increase their own intelligence through self-modification). So if you do a bunch of research and create a WBE, you have at some point figured out how to put that novel capability, not held by AI companies, into a computer.
But WBE research is AGI research.
| Are you going to also claim that the small primate, scaled up by , can’t be ML trained to be superhumanly intelligent? Ok. What about the chimp you do as a test? That can’t be made superhuman with scaling up and training? This kinda beggars belief, though maybe I’m missing something?
My claim is more that if we do get a chimp running at 10^3 and it somehow turns into ASI, I believe the resulting ASI is likely to be much more aligned, and we probably all get to live rather whatever we get scaling GPT6 etc.
I think that timelines for WBE if we’re trying to compete with the current AI companies in the free-market are substantially unlikely to happen (<5%). There’s a lot of tech missing for mass-manufacturing scanners etc, and to your point, I’m not sure the mouse-AI leads to immediate returns / revenue that you can go and get a bunch of VC funding for it anyway. Not enough to get to a human WBE fully funded by existing market mechanisms.
| I don’t think the AI companies have “zero problems” producing capabilities. I mean, that kinda contradicts the whole WBE plan, doesn’t it?
I think a human brain would have decisive headroom compared to current AIs. Potentially whatever is needed for RSI doesn’t exist in current AI or maybe never under the current regime, but it does exist in the human mind.
I also feel that a neuron-level scanning and emulation would not lead to any direct intelligence-algorithms breakthroughs. I think I could buy this argument pre LLMs. But we have current AIs where already know every single activation, and we don’t have any level of algorithmic understanding of what’s going on inside that leads us closer to any theory of intelligence.
Ok but I don’t think this is very relevant? My point is that, before you get something powerful & aligned, before that point, you got something that someone else could turn into something powerful & not-aligned by doing non-safe machine learning with your thing as a component / ingredient.
So then if a project gets a true full human WBE, before that point it created capabilities, not already present in an AI, but yes in a computer. At that point, those capabilities could be expropriated and trained on by AI capabilities researchers.