It requires only a little understanding of complex systems, and the nature of cognition, to realize that the recommendation “Go study math and theoretical computer science” is useless for understanding the Friendliness Problem.
For more details you can read one of the papers I have published on this topic. (See richardloosemore.com/papers). The short version of a long argument is that there are no known AI control (motivation) algorithms that are stable in the limit as the supergoal statements become abstract enough to drive a general intelligence. So, instead, we would expect their motivation systems to be structured as massive, weak constraint satisfaction engines. Such a weak constraint engine, although potentially stable (and friendly), is a complex system, so however friendly and stable it might be, those features will likely remain forever unprovable. Hence, mathematics is of no use.
The short version of a long argument is that there are no known AI control (motivation) algorithms that are stable in the limit as the supergoal statements become abstract enough to drive a general intelligence.
It requires only a little understanding of complex systems, and the nature of cognition, to realize that the recommendation “Go study math and theoretical computer science” is useless for understanding the Friendliness Problem.
For more details you can read one of the papers I have published on this topic. (See richardloosemore.com/papers). The short version of a long argument is that there are no known AI control (motivation) algorithms that are stable in the limit as the supergoal statements become abstract enough to drive a general intelligence. So, instead, we would expect their motivation systems to be structured as massive, weak constraint satisfaction engines. Such a weak constraint engine, although potentially stable (and friendly), is a complex system, so however friendly and stable it might be, those features will likely remain forever unprovable. Hence, mathematics is of no use.
At least, not yet?