given how many actual humans fail the existing Turing test miserably.
Are you referring to some specific thing here, or is this a general “people are dumb” sort of thing? If it’s the former, elaborate please, this sounds interesting!
Interestingly enough, I find numerous references to Clay and this incident, but very little in the way of the transcript it refers to. This was the best I could find (she is “Terminal 4”):
Judge 9: Are you familiar with Hamlet?
Terminal 4: The college kid who came home and found his mom had married the guy who murdered his dad just a little month before? You might say so.
[...]
Judge 1: What is your opinion on Shakespeare’s plays?
Terminal 4: That’s pretty general; would you be more specific? Otherwise, I’ll just say I like them.
Judge 1: Learning that you like them answers my question. Which of his plays is your favorite?
Terminal 4: Well, let’s see...Pericles.
Judge 1: Why is it your favorite?
Terminal 4: Because he was obviously called in to play-doctor somebody’s awful script. It was one of the few (maybe only two?) plays written with somebody else. It’s really rather an ucky play. What play do you like?
Judge 1: I did not understand your response. However, to answer your question, I do not have a favorite.
She does sound somewhat unhumanly to me in the latter excerpt you quoted. (But then again, so do certain Wikipedia editors that I’m pretty sure are human.)
It happens once in a while in online chat, where real people behave like chatterbots. Granted, with some probing it is possible to tell the difference, if the suspect stays connected long enough, but there were cases where I was not quite sure. In one case the person (I’m fairly sure it was a person) posted links to their own site and generic sentences like “there are many resources on ”. In another case the person was posting online news snippets and reacted with hostility to any attempts to engage, which is a perfect way to discourage Turing probing if you design a bot.
Imagine a normal test, perhaps a math test in a classroom. Someone knows math but falls asleep and doesn’t answer any of the questions. As a result, they fail the test. Would you say that the test can’t detect whether someone can do math?
Technically, that’s correct. The test doesn’t detect whether someone can do math; it detects whether they are doing math at the time. But it would be stupid to say “hey, you told me this tests if someone can do math! It doesn’t do that at all!” The fact that they used the words “can do” rather than “are doing at the time” is just an example of how human beings don’t use language like a machine, and objecting to that part is pointless.
Likewise, the fact that someone who acts like a chatterbot is detected as a computer by the Turing test does mean that the test doesn’t detect whether someone is a computer. It detects whether they are acting computerish at the time. But “this test detects whether someone is a computer” is how most people would normally describe that, even if that’s not technically accurate. It’s pointless to object that the test doesn’t detect whether someone is a computer on those grounds.
It doesn’t even test whether someone’s doing math at the time. I could be doing all kinds of math and, in consequence, fail the exam.
I would say, rather, that tests generally have implicit preconditions in order for interpretations of their results to be valid.
Standing on a scale is a test for my weight that presumes various things: that I’m not carrying heavy stuff, that I’m not being pulled away from the scale by a significant force, etc. If those presumptions are false and I interpret the scale readings normally, I’ll misjudge my weight. (Similarly, if I instead interpret the scale as a test of my mass, I’m assuming a 1g gravitational field, etc.)
Taking a math test in a classroom makes assumptions about my cognitive state—that I’m awake, trying to pass the exam, can understand the instructions, don’t have a gerbil in my pants, and so forth.
Are you referring to some specific thing here, or is this a general “people are dumb” sort of thing? If it’s the former, elaborate please, this sounds interesting!
I highly recommend his book Most Human Human, for an interesting perspective on how (most) humans pass the Turing Test.
Interestingly enough, I find numerous references to Clay and this incident, but very little in the way of the transcript it refers to. This was the best I could find (she is “Terminal 4”):
She does sound somewhat unhumanly to me in the latter excerpt you quoted. (But then again, so do certain Wikipedia editors that I’m pretty sure are human.)
Fascinating! Thank you, this is definitely going on my reading list.
It happens once in a while in online chat, where real people behave like chatterbots. Granted, with some probing it is possible to tell the difference, if the suspect stays connected long enough, but there were cases where I was not quite sure. In one case the person (I’m fairly sure it was a person) posted links to their own site and generic sentences like “there are many resources on ”. In another case the person was posting online news snippets and reacted with hostility to any attempts to engage, which is a perfect way to discourage Turing probing if you design a bot.
Imagine a normal test, perhaps a math test in a classroom. Someone knows math but falls asleep and doesn’t answer any of the questions. As a result, they fail the test. Would you say that the test can’t detect whether someone can do math?
Technically, that’s correct. The test doesn’t detect whether someone can do math; it detects whether they are doing math at the time. But it would be stupid to say “hey, you told me this tests if someone can do math! It doesn’t do that at all!” The fact that they used the words “can do” rather than “are doing at the time” is just an example of how human beings don’t use language like a machine, and objecting to that part is pointless.
Likewise, the fact that someone who acts like a chatterbot is detected as a computer by the Turing test does mean that the test doesn’t detect whether someone is a computer. It detects whether they are acting computerish at the time. But “this test detects whether someone is a computer” is how most people would normally describe that, even if that’s not technically accurate. It’s pointless to object that the test doesn’t detect whether someone is a computer on those grounds.
It doesn’t even test whether someone’s doing math at the time. I could be doing all kinds of math and, in consequence, fail the exam.
I would say, rather, that tests generally have implicit preconditions in order for interpretations of their results to be valid.
Standing on a scale is a test for my weight that presumes various things: that I’m not carrying heavy stuff, that I’m not being pulled away from the scale by a significant force, etc. If those presumptions are false and I interpret the scale readings normally, I’ll misjudge my weight. (Similarly, if I instead interpret the scale as a test of my mass, I’m assuming a 1g gravitational field, etc.)
Taking a math test in a classroom makes assumptions about my cognitive state—that I’m awake, trying to pass the exam, can understand the instructions, don’t have a gerbil in my pants, and so forth.