I am Issa Rice. https://issarice.com/
riceissa(Issa Rice)
Karma: 1,262
Is there a way to see all the users who predicted within a single “bucket” using the LW UI? Right now when I hover over a bucket, it will show all users if the number of users is small enough, but it will show a small number of users followed by ”...” if the number of users is too large. I’d like to be able to see all the users. (I know I can find the corresponding prediction on the Elicit website, but this is cumbersome.)
Ok. Since visiting your office hours is somewhat costly for me, I was trying to gather more information (about e.g. what kind of moral uncertainty or prior discussion you had in mind, why you decided to capitalize the term, whether this is something I might disagree with you on and might want to discuss further) to make the decision.
More generally, I’ve attended two LW Zoom events so far, both times because I felt excited about the topics discussed, and both times felt like I didn’t learn anything/would have preferred the info to just be a text dump so I could skim and move on. So I am feeling like I should be more confident that I will find an event useful now before attending.
Is any of the stuff around Moral Uncertainty real? I think it’s probably all fake, but if you disagree, let’s debate!
Can you say more about this? I only found this comment after a quick search.
Oh :0
I find the conjunction of your decision to have kids and your short AI timelines pretty confusing. The possibilities I can think of are (1) you’re more optimistic than me about AI alignment (but I don’t get this impression from your writings), (2) you think that even a short human life is worth living/net-positive, (3) since you distinguish between the time when humans lose control and the time when catastrophe actually happens, you think this delay will give more years to your child’s life, (4) your decision to have kids was made before your AI timelines became short. Or maybe something else I’m not thinking of? I’m curious to hear your thinking on this.
Does anyone know if the actual software and contents for the digital tutor are published anywhere? I tried looking in the linked report but couldn’t find anything like that there. I am feeling a bit skeptical that the digital tutor was teaching anything difficult. Right now I can’t even tell if the digital tutor was doing something closer to “automate teaching people how to use MS Excel” (sounds believable) vs “automate teaching people real analysis given AP Calculus level knowledge of math” (sounds really hard, unless the people are already competent at self-studying).
I am also interested in this.
There was “My Thoughts on Takeoff Speeds” by tristanm.
Thanks! I think I would have guessed that the optimal signup is around age 35-55 so this motivates me to dig closer into your model to see if I disagree with some parameter or modeling assumption (alternatively, I would be able to fix some mistaken intuition that I have). I’ve made a note to myself to come back to this when I have more free time.
There was a similar question a few months ago: Plans / prepping for possible political violence from upcoming US election?
Does anyone know how Brian Christian came to be interested in AI alignment and why he decided to write this book instead of a book about a different topic? (I haven’t read the book and looked at the Amazon preview but couldn’t find the answer there.)
Here is part of Paul’s definition of intent alignment:
In particular, this is the problem of getting your AI to try to do the right thing, not the problem of figuring out which thing is right. An aligned AI would try to figure out which thing is right, and like a human it may or may not succeed.
So in your first example, the partition seems intent aligned to me.
HCH is the result of a potentially infinite exponential process (see figure 1) and thereby, computationally intractable. In reality, we can not break down any task into its smallest parts and solve these subtasks one after another because that would take too much computation. This is why we need to iterate distillation and amplification and cannot just amplify.
In general your post talks about amplification (and HCH) as increasing the capability of the system and distillation as saving on computation/making things more efficient. But my understanding, based on this conversation with Rohin Shah, is that amplification is also intended to save on computation (otherwise we could just try to imitate humans). In other words, the distillation procedure is able to learn more quickly by training on data provided by the amplified system compared to just training on the unamplified system. So I don’t like the phrasing that distillation is the part that’s there to save on computation, because both parts seem to be aimed at that.
(I am making this comment because I want to check my understand with you or make sure you understand this point because it doesn’t seem to be stated in your post. It was one of the most confusing things about IDA to me and I’m still not sure I fully understand it.)
I still don’t understand how corrigibility and intent alignment are different. If neither implies the other (as Paul says in his comment starting with “I don’t really think this is true”), then there must be examples of AI systems that have one property but not the other. What would a corrigible but not-intent-aligned AI system look like?
I also had the thought that the implicative structure (between corrigibility and intent alignment) seems to depend on how the AI is used, i.e. on the particulars of the user/overseer. For example if you have an intent-aligned AI and the user is careful about not deploying the AI in scenarios that would leave them disempowered, then that seems like a corrigible AI. So for this particular user, it seems like intent alignment implies corrigibility. Is that right?
The implicative structure might also be different depending on the capability of the AI, e.g. a dumb AI might have corrigibility and intent alignment equivalent, but the two concepts might come apart for more capable AI.
IDA tries to prevent catastrophic outcomes by searching for a competitive AI that never intentionally optimises for something harmful to us and that we can still correct once it’s running.
I don’t see how the “we can still correct once it’s running” part can be true given this footnote:
However, I think at some point we will probably have the AI system autonomously execute the distillation and amplification steps or otherwise get outcompeted. And even before that point we might find some other way to train the AI in breaking down tasks that doesn’t involve human interaction.
After a certain point it seems like the thing that is overseeing the AI system is another AI system and saying that “we” can correct the first AI system seems like a confusing way to phrase this situation. Do you think I’ve understood this correctly / what do you think?
In the Alice/Bob diagrams, I am confused why the strategies are parameterized by the frequency of cooperation. Don’t these frequencies depend on what the other player does, so that the same strategy can have different frequencies of cooperation depending on who the other player is?
I am curious how good you think the conversation/facilitation was in the AI takeoff double crux between Oliver Habryka and Buck Shlegeris. I am looking for something like “the quality of facilitation at that event was X percentile among all the conversation facilitation I have done”.
Tyler Cowen would be distributing the money personally
According to Tyler Cowen’s blog post about the saliva test, this grant was made via Fast Grants. From the Fast Grants homepage:
Who will make grant decisions?
A panel of biomedical scientists will make funding recommendations to Emergent Ventures.The Fast Grants website does not mention Cowen, and his level of involvement is unclear to me. Some of the phrasing in your post like “Funded By Blogger’s Personal Fund” gave me the impression that Cowen was more involved in the decision-making process than I can find evidence for. I’m curious if you have more information on this.
Does this analysis take into account the fact that young people are most likely to die in ways that are unlikely to result in successful cryopreservation? If not, I’m wondering what the numbers look like if you re-run the simulation after taking this into account. As a young person myself, if I die in the next decade I think it is most likely to be from injury or suicide (neither of which seems likely to lead to successful cryopreservation), and this is one of the main reasons I have been cryocrastinating. See also this discussion.
Would this work across different countries (and if so how)? It seems like if one country implemented such a tax, the research groups in that country would be out-competed by research groups in other countries without such a tax (which seems worse than the status quo, since now the first AGI is likely to be created in a country that didn’t try to slow down AI progress or “level the playing field”).