Eric Drexler requested that I did not upload a recording to YouTube. Before the session, I compiled this document with most of the questions:
https://www.dropbox.com/s/i5oqix83wsfv1u5/Comprehensive_AI_Services_Q_A.pptx?dl=0
We did not get to post the last few questions. Are there any questions from this list you would like me to try to remember the answers to?
Recording of the session:
Thank you
Hi Howie,
Thank you for reminding me of these four documents. I had seen them, but I dismissed them early in the process. That might have been a mistake, and I’ll read them carefully now.
I think you did a great job at the interview. I describe one place where you could have pushed back more here: https://youtu.be/_kNvExbheNA?t=1376 You asked: ”...Assume that among the things that these narrow AIs are really good at doing, one of them is programming AI...”, and Ben Garfinkel made a broad answer about “doing science”.
I’d also like to pre-order from Denmark
Thank you. I’ve preordered now.
Today, I bought 20 shares in Gamestop / GME. I expect to lose money, and bought them as a hard-to-fake signal about willingness to coordinate and cooperate in the game-theoretic sense. This was inspired by Eliezer Yudkowsky’s post here: https://yudkowsky.medium.com/
In theory, Moloch should take all the ressources of someone following this strategy. In practice, Eru looks after her own, so I have the money to spare.
In the interview with AI Impacts, you said:
...examples of things that I’m optimistic about that they [people at MIRI] are super pessimistic about are like, stuff that looks more like verification...
Are you still optimistic? What do you consider the most promising recent work?
Thank you for your answer, and good luck with the Alignment Research Center.
I made my most strident and impolite presentation yet in the AISafety.com Reading Group last night. We were discussing “Conversation with Ernie Davis”, and I attacked this part:
“And once an AI has common sense it will realize that there’s no point in turning the world into paperclips...”
I described this as fundamentally mistaken and like an argument you’d hear from a person that had not read “Superintelligence”. This is ad hominem, and it pains me. However, I feel like the emperor has no clothes, and calling it out explicitly is important.
Yes, we are still running, though at a bi-weekly schedule. We will discuss Paul Christiano’s “Another (Outer) Alignment failure story” on the 8th of July.
How long time do you see between “1 AI clearly on track to Foom” and “First AI to actually Foom”? My weak guess is Eliezer would say “Probably quite little time”, but your model of the world requires the GWP to double over a 4 year period, and I’m guessing that period probably starts later than 2026.
I would be surprised if by 2027, I could point to an AI that for a full year had been on track to Foom, without Foom happening.
Paul Christiano makes a slightly different claim here: https://www.lesswrong.com/posts/7MCqRnZzvszsxgtJi/christiano-cotra-and-yudkowsky-on-ai-progress?commentId=AiNd3hZsKbajTDG2J
As I read the two claims:
With GPT-3 + 5 years of effort, a system could be built that would eventually Foom if allowed.
With GPT-3 + a serious effort, a system could be built that would clearly Foom if allowed.
I think the second could be made into a bet. I tried to operationalise it as a reply to the linked comment.
no, I think a human in a big animal body, with brain adapted to operate that body instead of our own, would beat a big animal straightforwardly
According to https://en.wikipedia.org/wiki/File:Brain-body_mass_ratio_for_some_animals_diagram.svg the hippopotamus would be the animal that “gained” the most from having a human brain, assuming that brain-body mass ratio indicates intelligence.
I could see the Ronaldo-brained hippopotamus winning by planning, cooperating and learning more from experience, but I’m not seeing a big advantage in a straight-up fight.
Absolutely fascinating link—strong upvote! Han et al.,2013 did not investigate motor control and motion planning, but I agree that human neural cells probably are just better, though possibly requiring more energy.
From martial arts, I’m convinced people have different innate levels of motor control and motion planning, and this helps nontrivially in fights. However, brains and muscles both require energy, and I’d generally give the advantage to the person with +1std muscles over the person with +1std motor control, assuming both are untrained.
I have a higher probability that Putin will launch the first nuke at Kiev. I think he might think that all other scenarios end with a Russian defeat and his personal untimely death. Russia is already a pariah state, and there is comparatively little for him to lose. Nuking Kiev would have the side-effect of making civilians flee urban centers in Ukraine, dramatically increasing the probability of a conventional Russian victory.
As a first approximation, if the food production falls to a level of X% of the required calories for the population, your probability of surviving is roughly X%.
Even a full counter-value nuclear exchange would not destroy all of our ability to produce food. Cities would be the primary targets, and they are net importers of food. Civilization might not even collapse with the removal of the 3000 largest cities in the western world.
It is possible that Putins political goals involve dismantling Ukraine along with the complete subjugation of the Ukrainian people. Nuclear weapons could thus have a desired political effect, in addition to their substantial practical effects.
According to this image, the performance is generally above the human average:
In the Paul-verse, we should expect that economic interests would quickly cause such models to be used for everything that they can be profitably used for. With better-than-average-human performance, that may well be a doubling of global GDP.
In the Eliezer-verse, the impact of such models on the GDP of the world will remain around $0, due to practical and regulatory constraints, right up until the upper line (“Human (Best)”) is surpassed for 1 particular task.
Good luck with the in-person AI Safety reading group. It sounds productive and fun.
For the past 2 years, I have been running the Skype-based AISafety.com Reading Group. You can see the material we have covered at https://aisafety.com/reading-group/ . Yesterday, Vadim Kosoy from MIRI gave a great presentation of his Learning-Theoretic Agenda: https://youtu.be/6MkmeADXcZg
Usually, I try to post a summary of the discussion to our Facebook group, but I’ve been unable to get a follow-on discussion going. Your summary/idea above is higher quality than what I post.
Please tell me if you have any ideas for collaboration between our reading groups, or if I can do anything else to help you :).