or that they will be robust to strong optimization at the time when AIs are capable of taking over. I think that’s probably wrong, because (1) LLMs have many more degrees of freedom in their internal representations than e.g. Inception, so the resulting optimized outputs are going to look even stranger
There has been some progress in robust ML since the days of DeepDream (2015).