Okay, I just did a deep-dive on the AI alignment problem and the Singularity on Wikipedia, and it will take me a while to digest all of that. My first impression is that it seems like an outlandish thing to worry about, but I am going to think about it more because I can easily imagine the situation reversed.
Plug in the numbers for current computing speeds, the current doubling time, and an estimate for the raw processing power of the human brain, and the numbers match in: 2021.
GPT-3 has some tens to hundreds of billion parameters and the human brain has 86 billion neurons, and I know it’s hand-waving because model parameters aren’t equivalent to human neurons, but—not bad! On the other hand, we’re seeing now what this numerical correspondence translates to in real life, and it’s interestingly different than what we I had imagined. AI is passing the Turing test, but the Turing test no longer feels like a hard line in the sand; it doesn’t seem to be testing what it was intended to test.
No one knows how to solve it, and it’s likely only rationalists could.
Understanding what, exactly, human values are would be a first step toward expressing it in AI. I hadn’t expected meta-ethics to get so applied.
...
You know what’s really odd? The word “singularity” appears only 33 times in the Sequences, mostly as “when I attended the Singularity Summit, someone said...” and such, without explanation of what it was. Most of the references were in the autobiographical section, which I didn’t read as deeply as the rest.
Figuring out what human values actually are is a pretty important part of the project. Though, we’d still have to figure out how to align it to them. Still, there is no end of use for applied meta-ethics here. You might also want to look into the Shard Theory subcommunity here - @TurnTrout and others are working on getting an understanding of how human values arise in the first place as “shards” of a much simpler optimization process in the human brain.
Okay, I just did a deep-dive on the AI alignment problem and the Singularity on Wikipedia, and it will take me a while to digest all of that. My first impression is that it seems like an outlandish thing to worry about, but I am going to think about it more because I can easily imagine the situation reversed.
Among the things I came across was that Eliezer was writing about this in 1996, and predicted
GPT-3 has some tens to hundreds of billion parameters and the human brain has 86 billion neurons, and I know it’s hand-waving because model parameters aren’t equivalent to human neurons, but—not bad! On the other hand, we’re seeing now what this numerical correspondence translates to in real life, and it’s interestingly different than what
weI had imagined. AI is passing the Turing test, but the Turing test no longer feels like a hard line in the sand; it doesn’t seem to be testing what it was intended to test.Understanding what, exactly, human values are would be a first step toward expressing it in AI. I hadn’t expected meta-ethics to get so applied.
...
You know what’s really odd? The word “singularity” appears only 33 times in the Sequences, mostly as “when I attended the Singularity Summit, someone said...” and such, without explanation of what it was. Most of the references were in the autobiographical section, which I didn’t read as deeply as the rest.
Figuring out what human values actually are is a pretty important part of the project. Though, we’d still have to figure out how to align it to them. Still, there is no end of use for applied meta-ethics here. You might also want to look into the Shard Theory subcommunity here - @TurnTrout and others are working on getting an understanding of how human values arise in the first place as “shards” of a much simpler optimization process in the human brain.