One screwup that you didn’t touch on was the 70%. 70% is the square root of 1⁄2, not 2. If it’s 2x as smart as its designers and the complexity class of smartness is square, then this new AI will be able to make one 40% smarter than it is, not 30% less smart. Imagine if the AI had been 9 times smarter than its designers… would its next generation have been 1⁄3 as smart as it started? It’s completely upside-down.
Two ‘Crawlviati’ attributions are inside the quotes.
You didn’t really call out certain objections as stronger than others. I would be surprised if giving up determinism was half as useful as giving up optimality. And changing the problem is huge. I think that, though this would not impact the actual strength of the argument, calling certain items out after the list before the next section would give it a rhetorical kick.
I sort of did touch on Naam’s screwup there; it doesn’t make sense to even talk about the next generation of AI being dumber, whether or not the ‘complexity’ is square, square root, log, or exponential or whether he calculated 70% right.
whups
I’m not sure which objections are stronger than others. Nondeterminism is probably less helpful to an AI than approximations, but is approximations more helpful than redefining problems? Computronium brute force? The possibility that P=NP and an AI can find a non-galactic algorithm for that? Weighing the stronger objections would require a much more precise total model of the constant factors and asymptotics and computational resources and possibilities for expansions.
I think redefining problems and approximation are both huge. I didn’t mean a complete ranking, just fleshing out and giving more life to certain elements after the list is done. Pointing out how big a deal they are. These are important failures in the argument. In a way it comes across as a kind of reverse Gish Gallop—you have a bunch of really really strong arguments, and by putting them in a list the impression is weakened.
One screwup that you didn’t touch on was the 70%. 70% is the square root of 1⁄2, not 2. If it’s 2x as smart as its designers and the complexity class of smartness is square, then this new AI will be able to make one 40% smarter than it is, not 30% less smart. Imagine if the AI had been 9 times smarter than its designers… would its next generation have been 1⁄3 as smart as it started? It’s completely upside-down.
Two ‘Crawlviati’ attributions are inside the quotes.
You didn’t really call out certain objections as stronger than others. I would be surprised if giving up determinism was half as useful as giving up optimality. And changing the problem is huge. I think that, though this would not impact the actual strength of the argument, calling certain items out after the list before the next section would give it a rhetorical kick.
I sort of did touch on Naam’s screwup there; it doesn’t make sense to even talk about the next generation of AI being dumber, whether or not the ‘complexity’ is square, square root, log, or exponential or whether he calculated 70% right.
whups
I’m not sure which objections are stronger than others. Nondeterminism is probably less helpful to an AI than approximations, but is approximations more helpful than redefining problems? Computronium brute force? The possibility that P=NP and an AI can find a non-galactic algorithm for that? Weighing the stronger objections would require a much more precise total model of the constant factors and asymptotics and computational resources and possibilities for expansions.
I think redefining problems and approximation are both huge. I didn’t mean a complete ranking, just fleshing out and giving more life to certain elements after the list is done. Pointing out how big a deal they are. These are important failures in the argument. In a way it comes across as a kind of reverse Gish Gallop—you have a bunch of really really strong arguments, and by putting them in a list the impression is weakened.