I do think that making an “approximate synthetic 2025 human newborn/fetus (mind)” that can be run on a server having 100x usual human thinking speed is almost certainly a finite problem, and one might get there by figuring out what structures are there in a fetus/newborn precisely enough, and it plausibly makes sense to focus particularly on structures which are more relevant to learning. If one were to pull this off, one might then further be able to have these synthetic fetuses grow up quickly into fairly normal humans and have them do stuff which ends the present period of (imo) acute x-risk. (And the development of thought continues after that, I think; I’ll say more that relates to this later.) While I do say in my post that making mind uploads is a finite problem, it might have been good to state also (or more precisely) that this type of thing is finite.
I certainly think that one can make a finite system such that one can reasonably think that it will start a process that does very much — like, eats the Sun, etc.. Indeed, I think it’s likely that by default humanity would unfortunately start a process that gets the Sun eaten this century. I think it is plausible there will be some people who will be reasonable in predicting pretty strongly that that particular process will get the Sun eaten. I think various claims about humans understanding some stuff about that process are less clear, though there is surely some hypothetical entity that could pretty deeply understand the development of that process up to the point where it eats the Sun.
Some things in my notes were written mostly with an [agent foundations]y interlocutor in mind, and I’m realizing now that some of these things could also be read as if I had some different interlocutor in mind, and that some points probably seem more incongruous if read this way.
I’ll now proceed to potential disagreements.
But there’s something else, which is a very finite legible learning algorithm that can automatically find all those things—the object-level stuff and the thinking strategies at all levels. The genome builds such an algorithm into the human brain. And it seems to work! I don’t think there’s any math that is forever beyond humans, or if it is, it would be for humdrum reasons like “not enough neurons to hold that much complexity in your head at once”.
Some ways I disagree or think this is/involves a bad framing:
If we focus on math and try to ask some concrete question, instead of asking stuff like “can the system eventually prove anything?”, I think it is much more appropriate to ask stuff like “how quickly can the system prove stuff?”. Like, brute-force searching all strings for being a proof of a particular statement can eventually prove any provable statement, but we obviously wouldn’t want to say that this brute-force searcher is “generally intelligent”. Very relatedly, I think that “is there any math which is technically beyond a human?” is not a good question to be asking here.
The blind idiot god that pretty much cannot even invent wheels (ie evolution) obviously did not put anything approaching the Ultimate Formula for getting far in math (or for doing anything complicated, really) inside humans (even after conditioning on specification complexity and computational resources or whatever), and especially not in an “unfolded form”[1], right? Any rich endeavor is done by present humans in a profoundly stupid way, right?[2] Humanity sorta manages to do math, but this seems like a very weak reason to think that [humans have]/[humanity has] anything remotely approaching an “ultimate learning algorithm” for doing math?[3]
The structures in a newborn [that make it so that in the right context the newborn grows into a person who (say) pushes the frontier of human understanding forward] and [which participate in them pushing the frontier of human understanding forward] are probably already really complicated, right? Like, there’s already a great variety of “ideas” involved in the “learning-relevant structures” of a fetus?
I think that the framing that there is a given fixed “learning algorithm” in a newborn, such that if one knew it, one would be most of the way there to understanding human learning, is unfortunate. (Well, this comes with the caveat that it depends on what one wants from this “understanding of human learning” — e.g., it is probably fine to think this if one only wants to use this understanding to make a synthetic newborn.) In brief, I’d say “gaining thinking-components is a rich thing, much like gaining technologies more generally; our ability to gain thinking-components is developing, just like our ability to gain technologies”, and then I’d point one to Note 3 and Note 4 for more on this.
I want to say more in response to this view/framing that some sort of “human learning algorithm” is already there in a newborn, even in the context of just the learning that a single individual human is doing. Like, a human is also importantly gaining components/methods/ideas for learning, right? For example, language is centrally involved in human learning, and language isn’t there in a fetus (though there are things in a newborn which create a capacity for gaining language, yes). I feel like you might want to say “who cares — there is a preserved learning algorithm in the brain of a fetus/newborn anyway”. And while I agree that there are very important things in the brain which are centrally involved in learning and which are fairly unchanged during development, I don’t understand what [the special significance of these over various things gained later] is which makes it reasonable to say that a human has a given fixed “learning algorithm”. An analogy: Someone could try to explain structure-gaining by telling me “take a random init of a universe with such and such laws (and look along a random branch of the wavefunction[4]) — in there, you will probably eventually see a lot of structures being created” — let’s assume that this is set up such that one in fact probably gets atoms and galaxies and solar systems and life and primitive entities doing math and reflecting (imo etc.). But this is obviously a highly unsatisfying “explanation” of structure-gaining! I wanted to know why/how protons and atoms and molecules form and why/how galaxies and stars and black holes form, etc.. I wanted to know about evolution, and about how primitive entities inventing/discovering mathematical concepts could work, and imo many other things! Really, this didn’t do very much beyond just telling me “just consider all possible universes — somewhere in there, structures occur”! Like, yes, I’ve been given a context in which structure-gaining happens, but this does very little to help me make sense of structure-gaining. I’d guess that knowing the “primordial human learning algorithm” which is there in a fetus is significantly more like knowing the laws of physics than your comment makes it out to be. If it’s not like that, I would like to understand why it’s not like that — I’d like to understand why a fetus’s learning-structures really deserve to be considered the “human learning algorithm”, as opposed to being seen as just providing a context in which wild structure-gaining can occur and playing some important role in this wild structure-gaining (for now).
to conclude: It currently seems unlikely to me that knowing a newborn’s “primordial learning algorithm” would get me close to understanding human learning — in particular, it seems unlikely that it would get me close understanding how humanity gains scientific/mathematical/philosophical understanding. Also, it seems really unlikely that knowing this “primordial learning algorithm” would get me close to understanding learning/technology-making/mathematical-understanding-gaining in general.[5]
One attempt to counter this: “but humans could reprogram into basically anything, including whatever better system for doing math there is!”. But conditional on this working out, the appeal of the claim that fetuses already have a load-bearing fixed “learning algorithm” is also defeated, so this counterargument wouldn’t actually work in the present context even if this claim were true.
That said, I could see an argument for a good chunk of the learning that most current humans are doing being pretty close to gaining thinking-structures which other people already have, from other people that already have them, and there is definitely something finite in this vicinity — like, some kind of pure copying should be finite (though the things humans are doing in this vicinity are of course more complicated than pure copying, there are complications with making sense of “pure copying” in this context, and also humans suck immensely (compared to what’s possible) even at “pure copying”).
Thank you for the comment!
First, I’d like to clear up a few things:
I do think that making an “approximate synthetic 2025 human newborn/fetus (mind)” that can be run on a server having 100x usual human thinking speed is almost certainly a finite problem, and one might get there by figuring out what structures are there in a fetus/newborn precisely enough, and it plausibly makes sense to focus particularly on structures which are more relevant to learning. If one were to pull this off, one might then further be able to have these synthetic fetuses grow up quickly into fairly normal humans and have them do stuff which ends the present period of (imo) acute x-risk. (And the development of thought continues after that, I think; I’ll say more that relates to this later.) While I do say in my post that making mind uploads is a finite problem, it might have been good to state also (or more precisely) that this type of thing is finite.
I certainly think that one can make a finite system such that one can reasonably think that it will start a process that does very much — like, eats the Sun, etc.. Indeed, I think it’s likely that by default humanity would unfortunately start a process that gets the Sun eaten this century. I think it is plausible there will be some people who will be reasonable in predicting pretty strongly that that particular process will get the Sun eaten. I think various claims about humans understanding some stuff about that process are less clear, though there is surely some hypothetical entity that could pretty deeply understand the development of that process up to the point where it eats the Sun.
Some things in my notes were written mostly with an [agent foundations]y interlocutor in mind, and I’m realizing now that some of these things could also be read as if I had some different interlocutor in mind, and that some points probably seem more incongruous if read this way.
I’ll now proceed to potential disagreements.
Some ways I disagree or think this is/involves a bad framing:
If we focus on math and try to ask some concrete question, instead of asking stuff like “can the system eventually prove anything?”, I think it is much more appropriate to ask stuff like “how quickly can the system prove stuff?”. Like, brute-force searching all strings for being a proof of a particular statement can eventually prove any provable statement, but we obviously wouldn’t want to say that this brute-force searcher is “generally intelligent”. Very relatedly, I think that “is there any math which is technically beyond a human?” is not a good question to be asking here.
The blind idiot god that pretty much cannot even invent wheels (ie evolution) obviously did not put anything approaching the Ultimate Formula for getting far in math (or for doing anything complicated, really) inside humans (even after conditioning on specification complexity and computational resources or whatever), and especially not in an “unfolded form”[1], right? Any rich endeavor is done by present humans in a profoundly stupid way, right?[2] Humanity sorta manages to do math, but this seems like a very weak reason to think that [humans have]/[humanity has] anything remotely approaching an “ultimate learning algorithm” for doing math?[3]
The structures in a newborn [that make it so that in the right context the newborn grows into a person who (say) pushes the frontier of human understanding forward] and [which participate in them pushing the frontier of human understanding forward] are probably already really complicated, right? Like, there’s already a great variety of “ideas” involved in the “learning-relevant structures” of a fetus?
I think that the framing that there is a given fixed “learning algorithm” in a newborn, such that if one knew it, one would be most of the way there to understanding human learning, is unfortunate. (Well, this comes with the caveat that it depends on what one wants from this “understanding of human learning” — e.g., it is probably fine to think this if one only wants to use this understanding to make a synthetic newborn.) In brief, I’d say “gaining thinking-components is a rich thing, much like gaining technologies more generally; our ability to gain thinking-components is developing, just like our ability to gain technologies”, and then I’d point one to Note 3 and Note 4 for more on this.
I want to say more in response to this view/framing that some sort of “human learning algorithm” is already there in a newborn, even in the context of just the learning that a single individual human is doing. Like, a human is also importantly gaining components/methods/ideas for learning, right? For example, language is centrally involved in human learning, and language isn’t there in a fetus (though there are things in a newborn which create a capacity for gaining language, yes). I feel like you might want to say “who cares — there is a preserved learning algorithm in the brain of a fetus/newborn anyway”. And while I agree that there are very important things in the brain which are centrally involved in learning and which are fairly unchanged during development, I don’t understand what [the special significance of these over various things gained later] is which makes it reasonable to say that a human has a given fixed “learning algorithm”. An analogy: Someone could try to explain structure-gaining by telling me “take a random init of a universe with such and such laws (and look along a random branch of the wavefunction[4]) — in there, you will probably eventually see a lot of structures being created” — let’s assume that this is set up such that one in fact probably gets atoms and galaxies and solar systems and life and primitive entities doing math and reflecting (imo etc.). But this is obviously a highly unsatisfying “explanation” of structure-gaining! I wanted to know why/how protons and atoms and molecules form and why/how galaxies and stars and black holes form, etc.. I wanted to know about evolution, and about how primitive entities inventing/discovering mathematical concepts could work, and imo many other things! Really, this didn’t do very much beyond just telling me “just consider all possible universes — somewhere in there, structures occur”! Like, yes, I’ve been given a context in which structure-gaining happens, but this does very little to help me make sense of structure-gaining. I’d guess that knowing the “primordial human learning algorithm” which is there in a fetus is significantly more like knowing the laws of physics than your comment makes it out to be. If it’s not like that, I would like to understand why it’s not like that — I’d like to understand why a fetus’s learning-structures really deserve to be considered the “human learning algorithm”, as opposed to being seen as just providing a context in which wild structure-gaining can occur and playing some important role in this wild structure-gaining (for now).
to conclude: It currently seems unlikely to me that knowing a newborn’s “primordial learning algorithm” would get me close to understanding human learning — in particular, it seems unlikely that it would get me close understanding how humanity gains scientific/mathematical/philosophical understanding. Also, it seems really unlikely that knowing this “primordial learning algorithm” would get me close to understanding learning/technology-making/mathematical-understanding-gaining in general.[5]
like, such that it is already there in a fetus/newborn and doesn’t have to be gained/built
I think present humans have much more for doing math than what is “directly given” by evolution to present fetuses, but still.
One attempt to counter this: “but humans could reprogram into basically anything, including whatever better system for doing math there is!”. But conditional on this working out, the appeal of the claim that fetuses already have a load-bearing fixed “learning algorithm” is also defeated, so this counterargument wouldn’t actually work in the present context even if this claim were true.
let’s assume this makes sense
That said, I could see an argument for a good chunk of the learning that most current humans are doing being pretty close to gaining thinking-structures which other people already have, from other people that already have them, and there is definitely something finite in this vicinity — like, some kind of pure copying should be finite (though the things humans are doing in this vicinity are of course more complicated than pure copying, there are complications with making sense of “pure copying” in this context, and also humans suck immensely (compared to what’s possible) even at “pure copying”).