“What is baffling to me is the vague idea that developing the theory of friendliness has any significant synergy with developing the theory of intelligence. ”
Makes perfect sense to me. ‘Friendliness’ requires being able to specify a very precise utility function to an optimiser. 99% of the job of figuring that out is the job of learning how to specify any utility function precisely. That’s a job that will have to be done by any AI developer.
“What is baffling to me is the vague idea that developing the theory of friendliness has any significant synergy with developing the theory of intelligence. ”
Makes perfect sense to me. ‘Friendliness’ requires being able to specify a very precise utility function to an optimiser. 99% of the job of figuring that out is the job of learning how to specify any utility function precisely. That’s a job that will have to be done by any AI developer.