Mathematics is essentially the art/science of making some statements about some kinds of values, and proving those statements. You need some working definition of “value”, “statement” and “proof”.
Set theory is a possible approach to define values, because you can use sets to emulate things like numbers, graphs, functions, etc. Logic is about making statements. Gödel theorem is related to proofs. If you want to teach a computer to do math, you probably want to work with these.
How is math related to alignment? Well, most things about alignment we don’t know how to solve yet—like, how to actually extract the human values. But assuming we solve this problem one day, we will probably also want the AI to reason correctly. That involves some math, for example using the Bayesian theorem. Should the AI try to improve itself, it better be good at reasoning about code and algorithms, which involves more math. And it may need to be good at reasoning about math itself.
This is not a 100% proof that at some moment the set theory will be needed (perhaps the AI can avoid this part of math, or can rediscover it on its own using some other parts of math), but given that it plays an important role in math today, and that math in general will be needed, it seems like a good idea for the AI researcher to know it.
(This is just my guess, I am not involved with MIRI.)
Thank you, but it is again like to say: “oh, to solve physics problem you need calculus. Calculus uses real numbers. The most elegant way to introduce real numbers is from rational numbers from natural numbers via Peano axiomatics. So let’s make physicists study Peano axiomatic, set theory and formal logic”.
In any area of math, you need some set theory and logic—but usually in the amount that can be covered in one-two pages.
Mathematics is essentially the art/science of making some statements about some kinds of values, and proving those statements. You need some working definition of “value”, “statement” and “proof”.
Set theory is a possible approach to define values, because you can use sets to emulate things like numbers, graphs, functions, etc. Logic is about making statements. Gödel theorem is related to proofs. If you want to teach a computer to do math, you probably want to work with these.
How is math related to alignment? Well, most things about alignment we don’t know how to solve yet—like, how to actually extract the human values. But assuming we solve this problem one day, we will probably also want the AI to reason correctly. That involves some math, for example using the Bayesian theorem. Should the AI try to improve itself, it better be good at reasoning about code and algorithms, which involves more math. And it may need to be good at reasoning about math itself.
This is not a 100% proof that at some moment the set theory will be needed (perhaps the AI can avoid this part of math, or can rediscover it on its own using some other parts of math), but given that it plays an important role in math today, and that math in general will be needed, it seems like a good idea for the AI researcher to know it.
(This is just my guess, I am not involved with MIRI.)
Thank you, but it is again like to say: “oh, to solve physics problem you need calculus. Calculus uses real numbers. The most elegant way to introduce real numbers is from rational numbers from natural numbers via Peano axiomatics. So let’s make physicists study Peano axiomatic, set theory and formal logic”.
In any area of math, you need some set theory and logic—but usually in the amount that can be covered in one-two pages.