So far nothing, was distracted by other stuff in my life. Yes, let’s chat! frombranestobrains@gmail.com
Valentin2026
Karma: 200
Mentorship in AGI Safety (MAGIS)
After the rest of the USA is destroyed the very unstable situation (especially taking into account how many people have guns) is quite likely. In my opinion countries (and remote parts of countries) that will not be under attack at all are much better
What to do if a nuclear weapon is used in Ukraine?
[Question] Would “Manhattan Project” style be beneficial or deleterious for AI Alignment?
Thank you for your research! First of all, I don’t expect the non-human parameter to give a clear power-law, since we need to add humans as well. Of course, close to singularity the impact of humans will be very small, but maybe we are not that close yet. Now for the details:
Compute:
1. Yes, Moore’s law was a quite steady exponential for quite a while, but we indeed should multiply it.
2. The graph shows just a five years period, and not the number of chips produced, but revenue. The five years period is too small for any conclusions, and I am not sure that fluctuations in revenue are not driven mainly by market price rather than by produced amount.
Data storage:
Yes, I saw that one before, seems more like they just draw a nice picture rather than real data.
General remarks:
I agree with the point that AGI appearance can be sufficiently random. I can see two mechanisms that potentially may make it less random. First, we may need a lot of computational resources, data storage etc. to create it, and as a lab or company reaches the threshold, it happens easily with already existing algorithms. Second, we may need a lot of digitalized data to train AGI, so the transition again happens only as we have that much data.
Lastly, notice that cthe reation of AGI is not a singularity in a mathematical sense yet. It will certainly accelerate our progress, but not to infinity, so if the data will predict for example singularity in 2030, it will likely mean AGI earlier than that.
How trustworthy would this prediction be? Depends on the amount of data and noise. If we have just 10-20 datapoints scattered all around the graph, so you can connect the dots in any way you like—not really. If, instead, we are lucky and the control parameter happened to be something easily measurable (something such that you can get just-in-time statistics, like the number of papers on arXiv right now, so we can get really a lot of data points) and the parameter continues to change as theory predicts—it would be a quite strong argument for the timeline.
It is not very likely that the control parameter will be that easily measurable and will obey power-law that good. I think it is a very high risk—very high gain project (very high gain, because if the prediction will be very clear it will be possible to persuade more people that the problem is important).
[Question] Impactful data science projects
You are making a good point. Indeed, the system that would reward authors and experts will be quite complicated, so I was thinking about it on a purely volunteering basis (so in the initial stages it is non-profit). Then, if the group of people willing to work on the project was formed, they may turn it into a business project. If the initial author of the idea is in the project, he may get something, otherwise, no—the idea is already donated, no donations back. I will make an update to the initial post to clarify this point.
As to your idea, I am totally not an expert in this field. Hopefully, we will find the experts for all our ideas (I also have a couple).
A proposed system for ideas jumpstart
Thank you very much, it does!
I think you answer is worth to be published as a separate post. It will be relevant for everyone who is teaching.
[Question] Grading scheme from Valentine Smith
It would be very interesting to look at the results of this experiment in more detail.
Yes, maybe I explained what I mean not very well; however, gjm (see commentaries below) seems to get it. The point is not that CFAR is very much like Lifespring (though I may have sounded like that), the point is that there are certain techniques (team spirit, deep emotional connections etc.) that are likely to be used in such workshops, that will most certainly make participants love workshop and organizers (and other participants) , but their effect on the participant’s life can be significantly weaker than their emotional change of mind. These techniques work sufficiently worse for the online workshops, so this was one of the reason I tried to understand why CFAR does not hold online workshops. Another reason was resentment towards CFAR for not doing it, for it would be much more convenient to me.
Is there any proven benefits of meditation retreats in comparison with regular meditation?
Ok, your point makes sense.
Basically, I am trying to figure out for myself if going to the workshop would be beneficial for me. I do believe that CFAR does not simply try to get as much money as possible. However, I am concerned that people after the workshop are strongly biased towards liking it not because it really helps, but because of psychological mechanisms akin to Lifespring. I am not saying that CFAR is doing it intentionally, it could just have been raised somehow on its own. Maybe these mechanisms are even beneficial to whatever CFAR is doing, but they definitely make evaluation harder.
“When I was talking to Valentine (head of curriculum design at the time) a while ago he said that the spirit is the most important thing about the workshop.”
Now, this already sounds a little bit disturbing and resembling Lifespring. Of course, the spirit is important, but I thought the workshop is going to arm us with instruments we can use in real life, not only in the emotional state of comradeship with like-minded rationalists.
I can understand your point, but I am not persuaded yet. Let me maybe clarify why. During the year and a half of COVID, the in-person workshops were not possible. During this time, there were people, who would strongly benefit from the workshop, and the workshop would be helpful at this time (for example, they were making a career choice). Some of them can allow private places for the time of the workshop. It seems that for them, during this time the online workshop would be certainly more beneficial than no workshop at all. Moreover, conducting at least one online workshop would be a good experiment that would give useful information. It is totally not obvious to me why the priors that “online workshop is useless or harmful, taking into account opportunity cost” are so high that this experiment should not be conducted.
Yes, I hope someone from CFAR can maybe explain it better to me.
It is a good justification for this behavior, but it does not seem to be the most rational choice. Indeed, one could specify that the participant of the online workshop must have a private space (own bedroom, office, hotel room, remote place in a park—whatever fits). I am pretty sure there is a significant number of people, who would prefer an online workshop to the offline one (especially when all offline are canceled due to COVID), and who have or can find a private space for the duration of the workshop. To say that we are not doing it because some people do not have privacy is like for the restaurant to stop offer meat to everyone because there are vegans among customers. Of course, online workshop is not for everyone, but there are people for whom it would work.
I agree that for some people physical contact (like hugs, handshaking etc.) indeed means a lot. However, it is not for everyone. Moreover, even if the online workshop is less effective due to lack of this spirit, is it indeed so ineffective that it is worse than no workshop at all? Finally, why just not to try? It sounds like a thing that should be tried at least one time, and if it fails—well, then we see that it fails.
Yes, I hope someone who attended CFAR (or even somehow related to it) would see this question and give their answer.
Yes, absolutely! We will open the application for mentee later