As I started saying last time, I really see it as something where it makes to adopt the beliefs of experts, as opposed to reasoning about it from first principles (assuming you in fact don’t have much expertise). If so, the question becomes which experts we should trust. Or, rather, how much weight to assign to various experts.
Piggybacking off of what westarted getting at in your previous post, there are lots of smart people in the world outside of the rationality community, and they aren’t taking AGI seriously. Maybe that’s for a good reason. Maybe they’re right and we’re wrong. Why isn’t Bill Gates putting at least 1% of his resources into it? Why isn’t Elon Musk? Terry Tao? Ray Dalio? Even people like Peter Theil and Vitalik Buterin who fund AI safety research only put a small fraction of their resources into it. It reminds me of this excerpt from HPMoR:
“Granted,” said Harry. “But Hermione, problem two is that not even wizards are crazy enough to casually overlook the implications of this. Everyone would be trying to rediscover the formula for the Philosopher’s Stone, whole countries would be trying to capture the immortal wizard and get the secret out of him—”
“It’s not a secret.” Hermione flipped the page, showing Harry the diagrams. “The instructions are right on the next page. It’s just so difficult that only Nicholas Flamel’s done it.”
“So entire countries would be trying to kidnap Flamel and force him to make more Stones. Come on, Hermione, even wizards wouldn’t hear about immortality and, and,” Harry Potter paused, his eloquence apparently failing him, “and just keep going. Humans are crazy, but they’re not that crazy!”
Personally, I put some weight in this, but not all that much (and all things considered I’m quite concerned). There are a lot of other low hanging fruit that they also don’t put resources into. Life extension, for example. Supposing they don’t buy that AGI is that big a deal, life extension is still a low hanging fruit, both from a selfish and an altruistic perspective. And so my model is just that, well, I expect that even the super smart people will flat out miss on lots of things.
I could be wrong about that though. Maybe my model is too pessimistic. I’m not enough of a student of history to really know, but it’d be interesting if anyone else could comment on what humanities track record has been with this sort of stuff historically. Or maybe some Tetlock followers would have something interesting to say.
PS: Oh, almost forgot: Robin Hanson doesn’t seem very concerned, and I place a good amount of weight on his opinions.
As I started saying last time, I really see it as something where it makes to adopt the beliefs of experts, as opposed to reasoning about it from first principles (assuming you in fact don’t have much expertise). If so, the question becomes which experts we should trust. Or, rather, how much weight to assign to various experts.
Piggybacking off of what we started getting at in your previous post, there are lots of smart people in the world outside of the rationality community, and they aren’t taking AGI seriously. Maybe that’s for a good reason. Maybe they’re right and we’re wrong. Why isn’t Bill Gates putting at least 1% of his resources into it? Why isn’t Elon Musk? Terry Tao? Ray Dalio? Even people like Peter Theil and Vitalik Buterin who fund AI safety research only put a small fraction of their resources into it. It reminds me of this excerpt from HPMoR:
Personally, I put some weight in this, but not all that much (and all things considered I’m quite concerned). There are a lot of other low hanging fruit that they also don’t put resources into. Life extension, for example. Supposing they don’t buy that AGI is that big a deal, life extension is still a low hanging fruit, both from a selfish and an altruistic perspective. And so my model is just that, well, I expect that even the super smart people will flat out miss on lots of things.
I could be wrong about that though. Maybe my model is too pessimistic. I’m not enough of a student of history to really know, but it’d be interesting if anyone else could comment on what humanities track record has been with this sort of stuff historically. Or maybe some Tetlock followers would have something interesting to say.
PS: Oh, almost forgot: Robin Hanson doesn’t seem very concerned, and I place a good amount of weight on his opinions.