I have nothing against having fun. But there is a moral difference between having fun playing video games, and having fun spending the (money equivalent of 100 saved lives) to climb Everest.
How do you define greatness? Would climbing something that no one else has ever climbed before, despite being attempted multiple times by professionals, fall under your definition?
Not sure if there is any greatness in climbing rocks at all.
Compare: finishing an unusually large pizza. It is dangerous, no one ever ate such a large pizza before, there is a lot of joy in eating a good pizza, and maybe your name will be mentioned in Wikipedia. But the feat is so pointless and banal, there is not greatness in it at all.
I find your “still decide to waste resources” argument poor. Where does the logic end? Should people have zero fun and live like beggars just to donate every last cent to fighting malaria? Why are you commenting on LessWrong when you could be out doing something altruistic?
Fair enough. I myself not sure where it should end. But some things are clear indicators of not caring enough for fellow humans. For example, installing a toilet made of pure gold, or spending 20k for being the 5000th person to climb Everest.
“Mountaineering is a rather banal adventure”—have you ever climbed a mountain? If so, how dangerous was it? I doubt those that have climbed K2 consider it banal.
Danger and banality are not opposite. E.g. speeding on the road is both banal and dangerous. Same for climbing K2. Sure, you get some animalistic pleasures: adrenaline, nice views, some respect from similarly wrong people. But that’s it.
“zero greatness in mountaineering”—there are still unclimbed peaks and new routes to already-summited peaks.
I don’t see how climbing a moderately big rock is a path to greatness.
“noble and altruistic ways to risk your life”—some people don’t care about nobility or altruism the way LW users do.
Well, they’re both wrong and are bad people.
Especially if they know that there are alternatives like donating for anti-mosquito nets, and yet they still decide to waste resources on mountaineering.
You’re right, science fiction and reading for pleasure clearly have the developmental benefits.
I myself was an avid consumer of science fiction, but most of it was junk, especially the Hollywood sci-fi. A much more carefully selected diet of fiction would’ve greatly benefited my development.
It’s so much easier to design a nutritious fiction diet these days, thanks to:
post-cyberpunk science fiction
tools like TV Tropes that allow checking a work for harmful tropes beforehand
Putin has at least two children, and he seems to care about them. For example, he gave one of his daughters Katerina Tikhonova several high-profile positions. According to the same source, Katerina reportedly gave him a grandchild.
Unless he is sure that his children will survive a nuclear apocalypse (which is unlikely), this could be a major factor for him.
Our current understanding of physics (and of our future capabilities) is so limited, I assume that our predictions on how the universe will behave trillions years from now are worthless.
I think we can safely postpone the entire question to the times after we achieve a decent understanding of physics, after we became much smarter, and after we can allow ourselves to invest some thousands of years of deep thought on the topic.
Quite a read, thanks!
But, at the risk of being the Captain Obvious, I must remind the readers that mountain climbing is stupid. At least until we have reliable mind uploading tech. Because, as the story illustrates, there is a very high risk of permadeath, and for no good reason.
In many readers, even the tragic stories like this trigger the deeply rooted desire for adventure, the desire for bravely risking your life to achieve greatness. But:
Mountaineering is a rather banal adventure. You’re climbing a rock. It’s not even the most interesting man-vs-nature adventure.
These days, there is zero greatness in mountaineering. There is no greatness in being the 5001th man who climbed Everest, not to mention less important rocks.
There are much more noble and altruistic ways to risk your life.
Compare: drinking gasoline to impress TikTok followers.
You’re right, the comparison is on purpose. Let’s assume that the cost of a new nuclear power plant is $1 bln. Why should one spend $1 bln on a nuclear plant instead of a solar / wind / storage factory?
In some situations, nuclear is likely the best option. E.g. settlements beyond the polar circle, or on Mars. Aside from that, I’m not sure if nuclear is a useful addition to the solar / wind / storage mix.
I don’t have a strong opinion on the issue, and the topic is new to me.
Both CI and Alcor accept small data storage items (e.g. a DVD / BluRay). But you have to be a member.
Kriorus can also do that, I think.
I have a similar digital mind backup project. No reason to wait for the proper mind uploading if we can already do some data transfer today. And the collected data could also be hugely beneficial for cryonics.
I can also store your data indefensibly, if you agree to store mine. For example, 700 MB.
I agree with the sentiment, but would like to be careful with interpreting the average human scores for AI benchmarks. Such scores are obtained under time constrains. And maybe not all human raters were sufficiently motivated to do their best. The ratings for top humans are more likely to be representative of the general human ability to do the task.
I would not update much on Foom from this. The paper’s results are only relevant to one branch of AI development (I would call it “enormous self-supervised DL”). There may be other branches where Foom is the default mode (e.g. some practical AIXI implementation), and which are under the radar for now.
But I agree, we now can be certain that AGI is indeed a matter of time. I also agree that it gives us a chance to experiment with a non-scary AGI first (e.g. some transformer descendant that beats humans on almost everything, but remains to be a one-way text-processing mincer).
Moreover, BIG bench makes the path to AGI shorter, as one can now measure progress towards it, and maybe even apply RL to directly maximize the score.
I think you’re right on both points.
Although I’m not sure if self-awareness is necessary to surpass humans at all cognitive tasks. I can imagine a descendant of GPT that completely fails the self-awareness benchmarks, yet is able to write the most beautiful poetry, conduct Nobel-level research in physics, and even design a superior version of itself.
Good catch! You’re right, if contributors continue to add harder and harder tasks to the benchmark, and do it fast enough, the benchmark could be forever ahead.
I expect that some day the benchmark will be frozen. And even if it’s not frozen, new tasks are added only a few times per month these days, thus it’s not impossible to solve its current version.
I agree, some future scaled-up versions of PaLM & Co may indeed be able to surpass top humans on BIG-Bench.
Could you explain the reasoning behind this claim? [ if a language model surpasses the top human score on it, the model is an AGI. ]
Ultimately, it’s the question of how we define “AGI”. One reasonable definition is “an AI that can do any cognitive task that humans can, and do it better than humans”.
Given its massive scope and diversity, BIG-bench seems to be a good enough proxy for “any cognitive task”.
Although I would use a stricter scoring than the average-across-tasks that was used in PaLM: the model must 1) beat top humans, 2) on each and every task of BIG-bench.
One could argue that the simple models like PaLM don’t have agency, goals, persistence of thought, self-awareness etc, and thus they can’t become the human-like AGI of science fiction. But it’s quite possible that such qualities are not necessary to do all cognitive tasks that humans can, but better.
A simple mechanistic algorithm can beat top humans in chess. Maybe another simple mechanistic algorithm can also beat top humans in science, poetry, AI engineering, strategic business management, childrearing, and in all other activities that make human intellectuals proud of themselves.
You’re right. And some of the existing tasks in the benchmark are way beyond the abilities of baseline humans (e.g. the image classification task where images are hex-encoded texts).
On the other hand, the organizers allowed the human testers to use any tool they want, including internet search, software etc. So, the measured top-human performance is the performance of humans augmented with technology.
I think an AI that can solve BIG-bench must be an AGI. But there could be an AGI that can’t solve BIG-bench yet.
The aforementioned Google’s Big-Bench paper is now publicly available: