Summary of potential problems spotted regarding the use of AlphaGoZero:
Complete visibility vs Incomplete visibility.
Almost complete experience (self-play) vs Once-only problems. Limits of attention.
Exploitation (a game match) vs Exploration (the real world).
Having one goal vs Having many conjunctive goals. Also, having utility maximisation goals vs Having target goals.
Who is affected by the adverse consequences (In a game vs In the real world)? - The problems of adversarial situation, and also of the cost externalising.
The related question of different timeframes.
Summary of clarifying questions:
Could you build a toy simulation? So we could spot assumptions and side-effects.
In which ways does it improve the existing social order? Will we still stay in mediocristan? Long feedback delay.
What is the scope of application of the idea (Global and central vs Local and diverse?)
Need concrete implementation examples. Any realistically imaginable practical implementation of it might not be so fine anymore, each time for different reasons.
Hello! Thanks for the prize announcement :)
Hope these observations and clarifying questions are of some help:
https://medium.com/threelaws/a-reply-to-aligned-iterated-distillation-and-amplification-problem-points-c8a3e1e31a30
Summary of potential problems spotted regarding the use of AlphaGoZero:
Complete visibility vs Incomplete visibility.
Almost complete experience (self-play) vs Once-only problems. Limits of attention.
Exploitation (a game match) vs Exploration (the real world).
Having one goal vs Having many conjunctive goals. Also, having utility maximisation goals vs Having target goals.
Who is affected by the adverse consequences (In a game vs In the real world)? - The problems of adversarial situation, and also of the cost externalising.
The related question of different timeframes.
Summary of clarifying questions:
Could you build a toy simulation? So we could spot assumptions and side-effects.
In which ways does it improve the existing social order? Will we still stay in mediocristan? Long feedback delay.
What is the scope of application of the idea (Global and central vs Local and diverse?)
Need concrete implementation examples. Any realistically imaginable practical implementation of it might not be so fine anymore, each time for different reasons.