What I had in mind wasn’t exactly the problem ‘there is more than one fixed point’, but more of ‘if you don’t understand what did you set up, you will end in a bad place’.
I think an example of a dynamic which we sort of understand and expect to reasonable by human standards is putting humans in a box and letting them deliberate about the problem for thousands of years. I don’t think this extends to eg. LLMs—if you tell me you will train a sequence of increasingly powerful GPT models and let them deliberate for thousands of human-speech-equivalent years and decide about the training of next-in-the sequence model, I don’t trust the process.
Thanks for the links!
What I had in mind wasn’t exactly the problem ‘there is more than one fixed point’, but more of ‘if you don’t understand what did you set up, you will end in a bad place’.
I think an example of a dynamic which we sort of understand and expect to reasonable by human standards is putting humans in a box and letting them deliberate about the problem for thousands of years. I don’t think this extends to eg. LLMs—if you tell me you will train a sequence of increasingly powerful GPT models and let them deliberate for thousands of human-speech-equivalent years and decide about the training of next-in-the sequence model, I don’t trust the process.
Fair enough.