Do you think factorization/debate would work for math? That is, do you think that to determine the truth of an arbitrarily complex mathematical argument with high probability, you or I could listen to a (well-structured) debate of superintelligences?
I ask because I’m not sure whether you think factorization just doesn’t work, or alignment-relevant propositions are much harder to factor than math arguments. (My intuition is that factorization works for math; I have little intuition for the difficulty of factoring alignment-relevant propositions but I suspect some are fully factorable.)
I be a lot more optimistic about it for math than for anything touching the real world.
Also, there are lots of real-world places where factorization is known to work well. Basically any competitive market, with lots of interchangeable products, corresponds to a good factorization of some production problem. Production lines, similarly, are good factorizations. The issue is that we can’t factor problems in general, i.e. there’s still lots of problems we can’t factor well, and using factorization as our main alignment strategy requires fairly general factorizability (since we have to factor all the sub-problems of alignment recursively, which is a whole lot of subproblems, and it only takes one non-human-factorable subproblem to mess it all up).
Do you think factorization/debate would work for math? That is, do you think that to determine the truth of an arbitrarily complex mathematical argument with high probability, you or I could listen to a (well-structured) debate of superintelligences?
I ask because I’m not sure whether you think factorization just doesn’t work, or alignment-relevant propositions are much harder to factor than math arguments. (My intuition is that factorization works for math; I have little intuition for the difficulty of factoring alignment-relevant propositions but I suspect some are fully factorable.)
I be a lot more optimistic about it for math than for anything touching the real world.
Also, there are lots of real-world places where factorization is known to work well. Basically any competitive market, with lots of interchangeable products, corresponds to a good factorization of some production problem. Production lines, similarly, are good factorizations. The issue is that we can’t factor problems in general, i.e. there’s still lots of problems we can’t factor well, and using factorization as our main alignment strategy requires fairly general factorizability (since we have to factor all the sub-problems of alignment recursively, which is a whole lot of subproblems, and it only takes one non-human-factorable subproblem to mess it all up).