I’ve looked at the Flare docs and been similarly unimpressed. Most of that is hindsight bias—knowing that the project remained (that I’m aware of) at the vaporware stage without delivering an actual language.
Some of the proposed language features are indeed attractive; the existing language that most closely resembles it is Javascript, which shares with LambdaMOO (mentioned in the Flare docs) the interesting feature of prototype inheritance (“parenting”).
Part of the negative impression comes from the docs being a catalog of proposed features, without a clear explanation of how each of those features participates in a coherent whole; it comes across as a “kitchen sink” approach to language design. Using XML as an underlying representation scheme being the most grating instance. The docs are long on how great Flare will be but short on programs written in Flare itself illustrating how and why the things you can do with Flare would be compelling to a programmer with a particular kind of problem to solve.
To give you an idea of my qualifications (or lack thereof) for evaluating such an effort: I’m an autodidact; I’ve never designed a new language, but I have fair implementation experience. I’ve written a LambdaMOO compiler targeting the Java VM as part of a commercial project (shipped), and attempted writing a Java VM in Java (never shipped, impratical without also writing a JIT, but quite instructive). That was back in 1998. These projects required learning quite a bit about language design and implementation.
It’s harder to comment on Eliezer’s other accomplishments—I’m rather impressed by the whole conceptual framework of FAI and CEV but it’s the kind of thing to be judged by the detailed drudge work required to make it all work afterward, rather than by the grand vision itself. I’m impressed (you have to be) with the AI box experiments.
I’m impressed (you have to be) with the AI box experiments.
I am confused and a little suspicious that he did a round with Carl Shulman as gatekeeper, where Carl let him out, whereas two others did not let him out. (If I misremembered someone please correct me.) Not sure exactly what about that feels suspicious to me, though...
The last three experiments had bigger (more than 2 orders of magnitude, I think) outside cash stakes. I suspect Russell and D. Alex may have been less indifferent about that than me, i.e. I think the record shows that Eliezer acquitted himself well with low stakes ($10, or more when the player is indifferent about the money) a few times, but failed with high stakes.
I think the record shows that Eliezer acquitted himself well with low stakes ($10, or more when the player is indifferent about the money) a few times, but failed with high stakes.
Which suggests to me that as soon as people actually feel a bit of real fear- rather than just role-playing- they become mostly immune to Eliezer’s charms.
With an actual boxed AI though, you probably want to let it out if it’s Friendly. It’s possibly the ultimate high stakes gamble. Certainly you have more to be afraid of than with a low stakes roleplay, but you also have a lot more to gain.
I’m impressed (you have to be) with the AI box experiments.
I’ve previously been rather scathing about those:
Those experiments are totally unscientific—and prove very little—except that some people like playing role-playing games where they act as a superintelligence, and then boast about how smart they are afterwards.
That sounds like you are trying to rouse anger, or expressing a personal dislike, but not much like an argument.
The AI-box experiments have the flavor of (and presumably are inspired by) the Turing test—you could equally have accused Turing at the time of being “unscientific” in that he had proposed an experiment that hadn’t even been performed and would not be for many years. Yes, they are a conceptual rather than a scientific experiment.
The point of the actual AI-box demonstration isn’t so much to “prove” something, in the sense of demonstrating a particular exploitable regularity of human behaviour that a putative UFAI could use to take over people’s brains over a text link (though that would be nice to have). Rather, it is that prior to the demonstration one would have assigned very little probability to the proposition “Eliezer role-playing an AI will win this bet”.
As such, I’d agree that they “prove little” but they do constitute evidence.
LOL, yes, that’s why it weights little. But, see, it still gets to considerably shift one’s expectations on the matter because it had a very low probability assigned to its happening, as per Conservation Of Expected Evidence. Let’s just say it counts as Rational Evidence, m’kay? Its merit is mostly to open places in Idea Space.
Eliezer Yudkowsky likes playing role-playing games where they act as a superintelligence, and then boast about how smart he is afterwards
Honestly, so do I. Have you ever played Genius The Transgression? Look, we all know he’s full of himself, he has acknowledged this himself, it’s a flaw of his, but it’s really really irrelevant to the quality of the experiment as evidence.
Where it does matter is that that trait and his militant, sneering, condescending atheism makes for awful, godawful PR. Nevertheless, I’ve heard he is working on that, and in his rationality book he will try to use less incendiary examples than in his posts here. Still, don’t expect it to go away too soon: he strikes me the sort of man who runs largely on pride and idealism and burning shounen passion: such an attitude naturally leads to some intellectual boisterousness: the expression of these symptoms can be toned down, but as long as the cause remains, they will show up every now and then. And if that cause is also what keeps him rollin’, I wouldn’t have it any other way.
LOL, yes, that’s why it weights little. But, see, it still gets to considerably shift one’s expectations on the matter because it had a very low probability assigned to its happening, as per Conservation Of Expected Evidence. Let’s just say it counts as Rational Evidence, m’kay?
Not m’kay. IIRC, it was complete junk science—an unrecorded, unverified role playing game with no witnesses.
I figure people should update about as much as they would if they were watching a Derren Brown show.
Thank you, but I could have done that myself, I meant “explain, in your own words if possible, what aspects of who this person is are relevant to the discussion”. So, he’s a Real Life version of The Mentalist. That is very cool. Why shouldn’t people get extract useful knowledge from his shows?
Er. Hmmm… Look, I don’t want to sound rude, but could you elaborate on that? As it is, your post provides me with no information at all, except for the fact that you seem to think it’s a bad idea...
Well, if you are not familiar with Derren Brown, perhaps my original comment is not for you. Derren is a magician who specialises in mind control. He has some of the skills he claims to have—but to get that across on TV is often difficult—since it is not easy to reassure the audience that you are not using stooges who are in on the tricks.
The evidence is materially better than ordinary anecdote because the fact of the experiment was published before results were available. And it’s a demonstration of reasonable possibility, not high probability. It’s n=5, but that’s materially better than nothing. In fact, taking some reasonable low probability of the human failure rate, such as 1%, the p-value is quite low as well, so it’s a statistically significant result.
I remember when Eliezer told people about the AI-box experiments he had not yet performed, and I predicted, with high confidence, that people would not “let him out of the box” and give him money; and I was wrong.
I still wonder if the conversations went something like this:
“If we say you let me out of the box, then people will take the risk of AI more seriously, possibly saving the world.”
“Oh. Okay, then.”
Eliezer said that no such trickery was involved. But, he would say that in either case.
I wouldn’t be persuaded to “let the AI out” by that argument. In fact, even after reading about the AI box experiments I still can’t imagine any argument that would convince me to let the AI out. As somebody not affiliated with SIAI at all, I think my somehow being persuaded would count for more evidence than, for instance Carl Shulman being persuaded. Unfortunately, because I’m not affiliated with the AI research community in general, I’m presumably not qualified to participate in an AI-box experiment.
I wouldn’t be persuaded to “let the AI out” by that argument. In fact, even after reading about the AI box experiments I still can’t imagine any argument that would convince me to let the AI out.
For some time now I suspect that the argument that convinced Carl Shulman and others was along the lines of acausal trade. See here, here and here. Subsequently I suspect that those who didn’t let the AI out of the box either didn’t understand the implications, haven’t had enough trust into the foundations and actuality of acausal trade, or were more like General Thud.
When Eliezer was doing them, the primary qualification was being willing to put up enough money to get Eliezer to do it. (I’m not criticizing him for this—it was a clever and interesting fundraising technique; and doing it for small sums would set a bad precedent.)
I still wonder if the conversations went something like this:
“If we say you let me out of the box, then people will take the risk of AI more seriously, possibly saving the world.”
“Oh. Okay, then.”
If he had said that to me, I would have asked what evidence there was that his putting the fear of machines into people would actually help anyone—except for him and possibly the members of his proposed “Fellowship of the AI”.
I’ve looked at the Flare docs and been similarly unimpressed. Most of that is hindsight bias—knowing that the project remained (that I’m aware of) at the vaporware stage without delivering an actual language.
Some of the proposed language features are indeed attractive; the existing language that most closely resembles it is Javascript, which shares with LambdaMOO (mentioned in the Flare docs) the interesting feature of prototype inheritance (“parenting”).
Part of the negative impression comes from the docs being a catalog of proposed features, without a clear explanation of how each of those features participates in a coherent whole; it comes across as a “kitchen sink” approach to language design. Using XML as an underlying representation scheme being the most grating instance. The docs are long on how great Flare will be but short on programs written in Flare itself illustrating how and why the things you can do with Flare would be compelling to a programmer with a particular kind of problem to solve.
To give you an idea of my qualifications (or lack thereof) for evaluating such an effort: I’m an autodidact; I’ve never designed a new language, but I have fair implementation experience. I’ve written a LambdaMOO compiler targeting the Java VM as part of a commercial project (shipped), and attempted writing a Java VM in Java (never shipped, impratical without also writing a JIT, but quite instructive). That was back in 1998. These projects required learning quite a bit about language design and implementation.
It’s harder to comment on Eliezer’s other accomplishments—I’m rather impressed by the whole conceptual framework of FAI and CEV but it’s the kind of thing to be judged by the detailed drudge work required to make it all work afterward, rather than by the grand vision itself. I’m impressed (you have to be) with the AI box experiments.
I am confused and a little suspicious that he did a round with Carl Shulman as gatekeeper, where Carl let him out, whereas two others did not let him out. (If I misremembered someone please correct me.) Not sure exactly what about that feels suspicious to me, though...
The record of AI box experiments (those involving Eliezer) is as follows:
Experiment 1, vs Nathan Russell—AI win
Experiment 2, vs David McFadzean—AI win
Experiment 3, vs Carl Shulman—AI win
Experiment 4, vs Russell Wallace—GK win
Experiment 5, vs D. Alex—GK win
The last three experiments had bigger (more than 2 orders of magnitude, I think) outside cash stakes. I suspect Russell and D. Alex may have been less indifferent about that than me, i.e. I think the record shows that Eliezer acquitted himself well with low stakes ($10, or more when the player is indifferent about the money) a few times, but failed with high stakes.
Which suggests to me that as soon as people actually feel a bit of real fear- rather than just role-playing- they become mostly immune to Eliezer’s charms.
With an actual boxed AI though, you probably want to let it out if it’s Friendly. It’s possibly the ultimate high stakes gamble. Certainly you have more to be afraid of than with a low stakes roleplay, but you also have a lot more to gain.
I’ve previously been rather scathing about those:
That sounds like you are trying to rouse anger, or expressing a personal dislike, but not much like an argument.
The AI-box experiments have the flavor of (and presumably are inspired by) the Turing test—you could equally have accused Turing at the time of being “unscientific” in that he had proposed an experiment that hadn’t even been performed and would not be for many years. Yes, they are a conceptual rather than a scientific experiment.
The point of the actual AI-box demonstration isn’t so much to “prove” something, in the sense of demonstrating a particular exploitable regularity of human behaviour that a putative UFAI could use to take over people’s brains over a text link (though that would be nice to have). Rather, it is that prior to the demonstration one would have assigned very little probability to the proposition “Eliezer role-playing an AI will win this bet”.
As such, I’d agree that they “prove little” but they do constitute evidence.
They constitute anecdotal evidence. Such evidence is usually considered to be pretty low-grade by scientists.
LOL, yes, that’s why it weights little. But, see, it still gets to considerably shift one’s expectations on the matter because it had a very low probability assigned to its happening, as per Conservation Of Expected Evidence. Let’s just say it counts as Rational Evidence, m’kay? Its merit is mostly to open places in Idea Space.
Honestly, so do I. Have you ever played Genius The Transgression? Look, we all know he’s full of himself, he has acknowledged this himself, it’s a flaw of his, but it’s really really irrelevant to the quality of the experiment as evidence.
Where it does matter is that that trait and his militant, sneering, condescending atheism makes for awful, godawful PR. Nevertheless, I’ve heard he is working on that, and in his rationality book he will try to use less incendiary examples than in his posts here. Still, don’t expect it to go away too soon: he strikes me the sort of man who runs largely on pride and idealism and burning shounen passion: such an attitude naturally leads to some intellectual boisterousness: the expression of these symptoms can be toned down, but as long as the cause remains, they will show up every now and then. And if that cause is also what keeps him rollin’, I wouldn’t have it any other way.
Not m’kay. IIRC, it was complete junk science—an unrecorded, unverified role playing game with no witnesses.
I figure people should update about as much as they would if they were watching a Derren Brown show.
Who’s Derren Brown?
Derren Brown
Thank you, but I could have done that myself, I meant “explain, in your own words if possible, what aspects of who this person is are relevant to the discussion”. So, he’s a Real Life version of The Mentalist. That is very cool. Why shouldn’t people get extract useful knowledge from his shows?
Well, go right ahead—if you have not encountered this sort of thing before.
Er. Hmmm… Look, I don’t want to sound rude, but could you elaborate on that? As it is, your post provides me with no information at all, except for the fact that you seem to think it’s a bad idea...
Well, if you are not familiar with Derren Brown, perhaps my original comment is not for you. Derren is a magician who specialises in mind control. He has some of the skills he claims to have—but to get that across on TV is often difficult—since it is not easy to reassure the audience that you are not using stooges who are in on the tricks.
Oh! Right. Actually that’s a very apt comparison!
The evidence is materially better than ordinary anecdote because the fact of the experiment was published before results were available. And it’s a demonstration of reasonable possibility, not high probability. It’s n=5, but that’s materially better than nothing. In fact, taking some reasonable low probability of the human failure rate, such as 1%, the p-value is quite low as well, so it’s a statistically significant result.
I remember when Eliezer told people about the AI-box experiments he had not yet performed, and I predicted, with high confidence, that people would not “let him out of the box” and give him money; and I was wrong.
I still wonder if the conversations went something like this:
“If we say you let me out of the box, then people will take the risk of AI more seriously, possibly saving the world.”
“Oh. Okay, then.”
Eliezer said that no such trickery was involved. But, he would say that in either case.
I wouldn’t be persuaded to “let the AI out” by that argument. In fact, even after reading about the AI box experiments I still can’t imagine any argument that would convince me to let the AI out. As somebody not affiliated with SIAI at all, I think my somehow being persuaded would count for more evidence than, for instance Carl Shulman being persuaded. Unfortunately, because I’m not affiliated with the AI research community in general, I’m presumably not qualified to participate in an AI-box experiment.
For some time now I suspect that the argument that convinced Carl Shulman and others was along the lines of acausal trade. See here, here and here. Subsequently I suspect that those who didn’t let the AI out of the box either didn’t understand the implications, haven’t had enough trust into the foundations and actuality of acausal trade, or were more like General Thud.
When Eliezer was doing them, the primary qualification was being willing to put up enough money to get Eliezer to do it. (I’m not criticizing him for this—it was a clever and interesting fundraising technique; and doing it for small sums would set a bad precedent.)
If he had said that to me, I would have asked what evidence there was that his putting the fear of machines into people would actually help anyone—except for him and possibly the members of his proposed “Fellowship of the AI”.