And Irving said it was a good question and he might need to get back to me on that.
I didn’t see the discussion, but it seems odd to me that a debate-aligner would grant the premise here—isn’t the whole idea of debate that informative responses do exist in very specific parts of the dialogue-tree? I don’t think the empirical case is as damning as you make it sound:
Mathematics mostly operates by the Scholastic Method. It has parts with easily demonstrable implications, but also other parts which are quite far from that, and it’s mostly fine.
And people do fail to recognise the right answer, justified with the right reasons, at times—but that is also not always the best argument (to them) for that right answer. You might have needed to present it differently, or attack the reasons people don’t accept the arguments first, or...etc. We don’t demand you do this stuff to get Science Genius Credits afterwards, but that also means you can find examples of people failing to appreciate the Perfect Science Genius, without that necessarily bearing on judging debates of Science-and-Pedagogy Geniuses.
I still don’t really expect this to work, but more so on priors.
I’m not sure I followed that. Are you saying something like: “Even for fallible humans, it seems likely there exists some argument good enough to persuade them of the truth, if you could somehow find that argument”?
Mathematics mostly operates by the Scholastic Method. It has parts with easily demonstrable implications, but also other parts which are quite far from that, and it’s mostly fine.
I think its useful to ask why math works, and the answer usually given is either 1) Extreme formality and rigor about what is being stated and why, or 2) lacking formality and rigor, physical intuitions, experiments, and predictions based on the intuitive argument.
Note both of these are not typical of debates, nor socratic methods.
| Mathematics mostly operates by the Scholastic Method.
I think this is why so many consider this the one true field. As far as I can tell, this applies to no other field other than mathematics.
And I think there was some hope in the past that alignment would be solved in some purely mathematical fashion aka formal alignment. But unfortunately, it doesn’t seem like maths can solve AI alignment because today’s AI isn’t built upon any mathematical theories to begin with.
I didn’t see the discussion, but it seems odd to me that a debate-aligner would grant the premise here—isn’t the whole idea of debate that informative responses do exist in very specific parts of the dialogue-tree? I don’t think the empirical case is as damning as you make it sound:
Mathematics mostly operates by the Scholastic Method. It has parts with easily demonstrable implications, but also other parts which are quite far from that, and it’s mostly fine.
And people do fail to recognise the right answer, justified with the right reasons, at times—but that is also not always the best argument (to them) for that right answer. You might have needed to present it differently, or attack the reasons people don’t accept the arguments first, or...etc. We don’t demand you do this stuff to get Science Genius Credits afterwards, but that also means you can find examples of people failing to appreciate the Perfect Science Genius, without that necessarily bearing on judging debates of Science-and-Pedagogy Geniuses.
I still don’t really expect this to work, but more so on priors.
I’m not sure I followed that. Are you saying something like: “Even for fallible humans, it seems likely there exists some argument good enough to persuade them of the truth, if you could somehow find that argument”?
I think its useful to ask why math works, and the answer usually given is either 1) Extreme formality and rigor about what is being stated and why, or 2) lacking formality and rigor, physical intuitions, experiments, and predictions based on the intuitive argument.
Note both of these are not typical of debates, nor socratic methods.
| Mathematics mostly operates by the Scholastic Method.
I think this is why so many consider this the one true field. As far as I can tell, this applies to no other field other than mathematics.
And I think there was some hope in the past that alignment would be solved in some purely mathematical fashion aka formal alignment. But unfortunately, it doesn’t seem like maths can solve AI alignment because today’s AI isn’t built upon any mathematical theories to begin with.