I think this extrapolates far from one example and I’m not sure the example applies all that well.
Old engines played ugly moves because of their limitations, not because playing ugly moves is a super power. They won anyway because humans cannot out calculate engines.
AlphaZero plays beautiful games and even todays standard engines don’t play ugly or dumb looking moves anymore. I think in the limit superior play will tend to be beautiful and elegant.
If there is a parallel between early super human chess and AGI takeover it will be that AGI uses less than brillant strategies that still work because of flawless or at least vastly superhuman execution. But these strategies will not look dump or incomprehensible.
Many high-level speedruns (and especially TAS runs) often look like some combination of completely stupid/insane/incomprehensible to casual players. Nevertheless, they work for the task they set out to do far more effectively than trying to beat the game quickly with “casual strats” would get you.
I think seeing a sufficiently smart AI doing stuff in the real world would converge to looking a lot like that from our POV.
Counterpoint while working within the metaphor: early speedruns usually look like exceptional runs of the game played casually, with a few impressive/technical/insane moves thrown in.
Counterpoint: such strategies typically requires lots of iteration with perfect emulation of the system targeted to develop (I’m thinking in particular of glitch exploitation). Robust strategies might appear more “elegant.”
I think this extrapolates far from one example and I’m not sure the example applies all that well.
Old engines played ugly moves because of their limitations, not because playing ugly moves is a super power. They won anyway because humans cannot out calculate engines.
AlphaZero plays beautiful games and even todays standard engines don’t play ugly or dumb looking moves anymore. I think in the limit superior play will tend to be beautiful and elegant.
If there is a parallel between early super human chess and AGI takeover it will be that AGI uses less than brillant strategies that still work because of flawless or at least vastly superhuman execution. But these strategies will not look dump or incomprehensible.
I’d provide a counterexample analogy: speedruns.
Many high-level speedruns (and especially TAS runs) often look like some combination of completely stupid/insane/incomprehensible to casual players. Nevertheless, they work for the task they set out to do far more effectively than trying to beat the game quickly with “casual strats” would get you.
I think seeing a sufficiently smart AI doing stuff in the real world would converge to looking a lot like that from our POV.
Counterpoint while working within the metaphor: early speedruns usually look like exceptional runs of the game played casually, with a few impressive/technical/insane moves thrown in.
Counterpoint: such strategies typically requires lots of iteration with perfect emulation of the system targeted to develop (I’m thinking in particular of glitch exploitation). Robust strategies might appear more “elegant.”