I’ve been critical about your “theory of change” in the past, but regardless of that, I think the arguments in this post are completely right. Things really are this bad. And yeah, joining big labs to work on alignment is especially not helping.
A lot of people have pointed out the CS Lewis quote which is perfect for the occasion:
“At first, of course,” said Filostrato, “the power will be confined to a number—a small number—of individual men. Those who are selected for eternal life.”
“And you mean,” said Mark, “it will then be extended to all men?”
“No,” said Filostrato. “I mean it will then be reduced to one man.”
Ah, yes. That Hideous Strength. A strange novel, certainly.
I’ve long felt that it was about the Singularity, as seen from a primarily biological perspective. You see the dreams of power and transcendence, the early partial successes, and the inevitable corruption of nearly all the people involved. There’s even a discussion of s-risks. The ending is positive because C.S. Lewis wanted to tell an entertaining story, and because his religion encouraged him to believe in impossible last-minute grace.
I may have referred to various AI labs as “the National Institute for Coordinated Experiments” in the group chat. I suspect that they employ more than one Filostrato, Wither or Frost.
Google once prided itself on “Don’t be Evil,” and they even sometimes lived up to it, to some degree. But money and power have corrupted them pretty thoughly, over stakes no higher than some ad revenue. And one of the arguments of That Hideous Strength is that people who push for a Singularity are unusually corruptible, because the temptations are so great. And, frankly, because they would need to be unspeakably arrogant and reckless to try.
I’ve been critical about your “theory of change” in the past, but regardless of that, I think the arguments in this post are completely right. Things really are this bad. And yeah, joining big labs to work on alignment is especially not helping.
A lot of people have pointed out the CS Lewis quote which is perfect for the occasion:
Ah, yes. That Hideous Strength. A strange novel, certainly.
I’ve long felt that it was about the Singularity, as seen from a primarily biological perspective. You see the dreams of power and transcendence, the early partial successes, and the inevitable corruption of nearly all the people involved. There’s even a discussion of s-risks. The ending is positive because C.S. Lewis wanted to tell an entertaining story, and because his religion encouraged him to believe in impossible last-minute grace.
I may have referred to various AI labs as “the National Institute for Coordinated Experiments” in the group chat. I suspect that they employ more than one Filostrato, Wither or Frost.
Google once prided itself on “Don’t be Evil,” and they even sometimes lived up to it, to some degree. But money and power have corrupted them pretty thoughly, over stakes no higher than some ad revenue. And one of the arguments of That Hideous Strength is that people who push for a Singularity are unusually corruptible, because the temptations are so great. And, frankly, because they would need to be unspeakably arrogant and reckless to try.