could be that some kinds of “WBE research”, especially ones such as connectomics that don’t try to reverse engineer the brain’s learning algorithm, are net-helpful. See Byrnes 2023: “Connectomics seems great from an AI x-risk perspective”. I a little bit believe this regarding connectomics. I don’t strongly believe this about connectomics or about any research, a priori, because either it doesn’t lead to true, extremely helpful, human WBEs, or else it does and it’s powerful and therefore it throws off powerful PBEs on the way there.
How do you feel about PBE type projects like this specifically for the purposes of solving alignment?
If they make it significantly easier for someone to make an actual PBE, that’s bad. Publishing a connectome probably makes that significantly easier.
While you were trying to get dregs of information for alignment from the mouse brain (which I don’t think will work much), someone else could take the mouse brain emulation, and multiply the mouse cortex by . Then they could use machine learning (since, unlike you, they are crazy and reckless) to train their weird hypermouse to be highly performant at many tasks; they can try whatever interconnectivity hyperparameters they want, biorealistic or not, and they can try any neural learning algorithms they want, because they’re crazy and don’t care about AI capabilities risk. They seem to have a lot of tools for expropriating this stuff, which you did help to make available.
I guess a relevant aspect here is that I don’t think the connectome would actually be that helpful for making a mouse-y intelligence, since the hard and important part of mouse-y intelligence is the cortex, and there are already tens of thousands of papers on the mouse cortex from decades of study, including the MICrONS dataset, and I don’t really see what new useful things you’d learn about the cortex from a full connectome given what’s already known. I think the cortex is mostly about learning, and a connectome gives you a snapshot, not a learning trajectory. (I don’t claim it would be zero helpful, just that it would be only marginally helpful, whereas it would be strongly helpful for some alignment-relevant questions.)
I agree with this. I doubt very much that connectomics and brain emulation research is going to help very much with AI capabilities/intelligence. I think there is a higher probability that this type of research could help with AI alignment, but that it is still probably unlikely to help very much there, e.g. because progress for non-brain inspired AI seems to be progressing so rapidly.
(My personal interest in WBE is primarily because I think it could be a method for revival following structural brain preservation, so I have learned some about the space, but I’m certainly not an expert in it. I expect that this mostly won’t be done until we are past some serious AI transitions, or will never be done if we don’t survive them.)
How do you feel about PBE type projects like this specifically for the purposes of solving alignment?
https://ifp.org/mapping-the-brain-for-alignment/”
If they make it significantly easier for someone to make an actual PBE, that’s bad. Publishing a connectome probably makes that significantly easier.
While you were trying to get dregs of information for alignment from the mouse brain (which I don’t think will work much), someone else could take the mouse brain emulation, and multiply the mouse cortex by
(I played some role in that IFP essay being created … which btw got cited by a white house document on funding priorities so hope it’s wise 🤞)
Mostly see my other comment.
I guess a relevant aspect here is that I don’t think the connectome would actually be that helpful for making a mouse-y intelligence, since the hard and important part of mouse-y intelligence is the cortex, and there are already tens of thousands of papers on the mouse cortex from decades of study, including the MICrONS dataset, and I don’t really see what new useful things you’d learn about the cortex from a full connectome given what’s already known. I think the cortex is mostly about learning, and a connectome gives you a snapshot, not a learning trajectory. (I don’t claim it would be zero helpful, just that it would be only marginally helpful, whereas it would be strongly helpful for some alignment-relevant questions.)
I agree with this. I doubt very much that connectomics and brain emulation research is going to help very much with AI capabilities/intelligence. I think there is a higher probability that this type of research could help with AI alignment, but that it is still probably unlikely to help very much there, e.g. because progress for non-brain inspired AI seems to be progressing so rapidly.
(My personal interest in WBE is primarily because I think it could be a method for revival following structural brain preservation, so I have learned some about the space, but I’m certainly not an expert in it. I expect that this mostly won’t be done until we are past some serious AI transitions, or will never be done if we don’t survive them.)
(See my response to your other comment.)