the claim that “the best chess programs are still not always beating grandmasters” is false
My impression is that draws can still occasionally occur against grandmasters. Your point about handicaps is a very good one.
Top-tier grandmasters could still reliably beat computers all until early-to-mid nineties, and the period of rough equivalence between top grandmasters and top computers lasted for only a few years—from the development of Deep Blue in 1996 to sometime in the early 2000s. And even then, the differences between human and machine skills were very great in different aspects of the game—computers were far better in tactical calculations, but inferior in long-term positional strategy, so there was never any true equivalence.
That’s another good point. However, it does get into the question of what we mean by equivalent and what metric you are using. Almost all technologies (not just computer technologies) accomplish their goals in a way that is very different than how humans do. That means that until the technology is very good there will almost certainly be a handful of differences between what the human does well and what the computer does well.
It seems in the context of the original conversation, whether the usual pattern of technological advancement is evidence against Clippy’s narrative, the relevant era to compare Clippy to in this context would be the long period where computers could beat the vast majority of chess players but sitll sometimes lost to grandmasters. That period lasted from the late 1970s to a bit over 2000. By analogy, Clippy would be in the period where it is smarter than most humans (I think we’d tentatively agree that that appears to be the case) but not so smart as to be of vastly more intelligent than humans. Using the Chess example, that period of time could plausibly last quite some time.
Also, Clippy’s intelligence may be limited in what areas it can handle.There’s a natural plateau for the natural language problem in that once it is solved that specific aspect won’t see substantial advancement from casual conversation. (There’s also a relevant post that I can’t seem to find where Eliezer discussed the difficulty of evaluating the intelligence of people that are much smarter than you.) If that’s the case, then Clippy is plausibly at the level where it can handle most forms of basic communication but hasn’t handled other levels of human processing to the point where it has generally become even with the smartest humans. For example, there’s evidence for this in that Clippy has occasionally made errors of reasoning and has demonstrated that it has a very naive understanding of human social interaction protocols.
My impression is that draws can still occasionally occur against grandmasters.
And I can get a draw (more than occasionally) against computer programs I have almost no hope of ever winning against. Draws are easy if you do not try to win.
From what I know, at grandmaster level, it is generally considered to be within the white player’s power to force the game into a dead-end drawn position, leaving the black no sensible alternative at any step. This is normally considered cowardly play, but it’s probably the only way a human could hope for even a draw against a top computer these days.
With black pieces, I doubt that even the most timid play would help against a computer with an extensive opening book, programmed to steer the game into maximally complicated and uncertain positions at every step. (I wonder if anyone has looked at the possibility of teaching computers Mikhail Tal-style anti-human play, where they would, instead of calculating the most sound and foolproof moves, steer the game into mind-boggling tactical complications where humans would get completely lost?) In any case, I am sure that taking any initiative would be a suicidal move against a computer these days.
(Well, there is always a very tiny chance that the computer might blunder.)
My impression is that draws can still occasionally occur against grandmasters. Your point about handicaps is a very good one.
That’s another good point. However, it does get into the question of what we mean by equivalent and what metric you are using. Almost all technologies (not just computer technologies) accomplish their goals in a way that is very different than how humans do. That means that until the technology is very good there will almost certainly be a handful of differences between what the human does well and what the computer does well.
It seems in the context of the original conversation, whether the usual pattern of technological advancement is evidence against Clippy’s narrative, the relevant era to compare Clippy to in this context would be the long period where computers could beat the vast majority of chess players but sitll sometimes lost to grandmasters. That period lasted from the late 1970s to a bit over 2000. By analogy, Clippy would be in the period where it is smarter than most humans (I think we’d tentatively agree that that appears to be the case) but not so smart as to be of vastly more intelligent than humans. Using the Chess example, that period of time could plausibly last quite some time.
Also, Clippy’s intelligence may be limited in what areas it can handle.There’s a natural plateau for the natural language problem in that once it is solved that specific aspect won’t see substantial advancement from casual conversation. (There’s also a relevant post that I can’t seem to find where Eliezer discussed the difficulty of evaluating the intelligence of people that are much smarter than you.) If that’s the case, then Clippy is plausibly at the level where it can handle most forms of basic communication but hasn’t handled other levels of human processing to the point where it has generally become even with the smartest humans. For example, there’s evidence for this in that Clippy has occasionally made errors of reasoning and has demonstrated that it has a very naive understanding of human social interaction protocols.
And I can get a draw (more than occasionally) against computer programs I have almost no hope of ever winning against. Draws are easy if you do not try to win.
From what I know, at grandmaster level, it is generally considered to be within the white player’s power to force the game into a dead-end drawn position, leaving the black no sensible alternative at any step. This is normally considered cowardly play, but it’s probably the only way a human could hope for even a draw against a top computer these days.
With black pieces, I doubt that even the most timid play would help against a computer with an extensive opening book, programmed to steer the game into maximally complicated and uncertain positions at every step. (I wonder if anyone has looked at the possibility of teaching computers Mikhail Tal-style anti-human play, where they would, instead of calculating the most sound and foolproof moves, steer the game into mind-boggling tactical complications where humans would get completely lost?) In any case, I am sure that taking any initiative would be a suicidal move against a computer these days.
(Well, there is always a very tiny chance that the computer might blunder.)