most recent Singularity Institute papers are neither empirical (“we did experiment X, these are the results”) or mathematical (“if you assume A, B, and C, then D and E follow”). Rather, they are philosophical, like Paul Graham’s essays. I honestly can’t think of a single instance where I was convinced of an informal, philosophical argument through an academic paper. Books, magazines, blog posts—sure, but papers just don’t seem to be a thing.
Hopefully there will be some mathematical results coming out at some point, it’s the only way to make any real progress toward the stated objective of provably friendly AI.
First, “Well, at some point somebody’s going to have to figure out” how to partition the issue into manageable chunks. Like, what would be some of the very first steps?
Invent a model of a sequential learning algorithm that has access to its own source code and can rewrite it in some way—in short, the model should consider itself part of “its world,” in contrast to the way AIXI isn’t. Giving it an explicit reward channel is probably a bad idea.
Prove that algorithm learns sequences in some optimal or nearly optimal sense.
Develop approximate algorithms that are actually computable if the resulting algorithm fails to be.
This is so far my expertise domain that I hesitate to open up any of these black boxes any further.
You have the essential training in math research. This makes you at least as qualified as probably anyone on the SI staff.
You flatter me. My training is in medical imaging and inverse problems, not logic and machine learning. I’ve probably spent a total of eight hours thinking about sequence learning algorithms in my life.
Hopefully there will be some mathematical results coming out at some point, it’s the only way to make any real progress toward the stated objective of provably friendly AI.
Well, at some point somebody’s going to have to figure out (formally, mathematically) whatever it is that is meant by “naturalistic” AIXI.
First, “Well, at some point somebody’s going to have to figure out” how to partition the issue into manageable chunks. Like, what would be some of the very first steps?
Invent a model of a sequential learning algorithm that has access to its own source code and can rewrite it in some way—in short, the model should consider itself part of “its world,” in contrast to the way AIXI isn’t. Giving it an explicit reward channel is probably a bad idea.
Prove that algorithm learns sequences in some optimal or nearly optimal sense.
Develop approximate algorithms that are actually computable if the resulting algorithm fails to be.
This is so far my expertise domain that I hesitate to open up any of these black boxes any further.
What would be the first step in doing that? Alternatively, is it really the first step?
You have the essential training in math research. This makes you at least as qualified as probably anyone on the SI staff.
You flatter me. My training is in medical imaging and inverse problems, not logic and machine learning. I’ve probably spent a total of eight hours thinking about sequence learning algorithms in my life.