Thank you, wonderful series!
Just Learning
After the rest of the USA is destroyed the very unstable situation (especially taking into account how many people have guns) is quite likely. In my opinion countries (and remote parts of countries) that will not be under attack at all are much better
I agree with the point about the continuous ability to suffer rather than a threshold. I totally agree that there is no objective answer, we can’t measure sufferings. The problem is, however, that it leaves a practical question that is not clear how to solve, namely how we should treat other animals and our code.
I like the idea. Basically, you suggest taking the functional approach and advance it. What do you think can be this type of process?
Sorry, I didn’t get what do you mean by “non-dominant political controllership”, can you rephrase it?
Of course, placebo is useful from the evolutionary point of view, and it is a subject of quite a lot of research. (Main idea—it is energetically costly to have your immune system always at high alert, so you boost it in particular moments, correlating with pleasure, usually from eating/drinking/sex, which is when germs usually get to the body. If interested, I will find the link to the research paper where it is discussed. ).
I am afraid I still fail to explain what I mean. I do not try to deduce from the observation that we are in a simulation, I don’t think it is possible (unless simulators decide to allow it).
I am trying to see how the belief that we are in simulation with benevolent simulators can change my subjective experience. Notice, I can’t just trick myself to believe only because it is healthy to believe. This is why I needed all this theory above—to show that benevolent simulators are indeed highly likely. Then, and only then, I can hope for the placebo effect (or for real intervention masquerading under placebo effect), because now I believe that it may work. If I could just make myself to believe in whatever I needed, of course I would not need all these shenanigans—but, after being faithful LW reader for a while, it is really hard, if possible at all.
Thank you for your research! First of all, I don’t expect the non-human parameter to give a clear power-law, since we need to add humans as well. Of course, close to singularity the impact of humans will be very small, but maybe we are not that close yet. Now for the details:
Compute:
1. Yes, Moore’s law was a quite steady exponential for quite a while, but we indeed should multiply it.
2. The graph shows just a five years period, and not the number of chips produced, but revenue. The five years period is too small for any conclusions, and I am not sure that fluctuations in revenue are not driven mainly by market price rather than by produced amount.
Data storage:
Yes, I saw that one before, seems more like they just draw a nice picture rather than real data.
General remarks:
I agree with the point that AGI appearance can be sufficiently random. I can see two mechanisms that potentially may make it less random. First, we may need a lot of computational resources, data storage etc. to create it, and as a lab or company reaches the threshold, it happens easily with already existing algorithms. Second, we may need a lot of digitalized data to train AGI, so the transition again happens only as we have that much data.
Lastly, notice that cthe reation of AGI is not a singularity in a mathematical sense yet. It will certainly accelerate our progress, but not to infinity, so if the data will predict for example singularity in 2030, it will likely mean AGI earlier than that.
How trustworthy would this prediction be? Depends on the amount of data and noise. If we have just 10-20 datapoints scattered all around the graph, so you can connect the dots in any way you like—not really. If, instead, we are lucky and the control parameter happened to be something easily measurable (something such that you can get just-in-time statistics, like the number of papers on arXiv right now, so we can get really a lot of data points) and the parameter continues to change as theory predicts—it would be a quite strong argument for the timeline.
It is not very likely that the control parameter will be that easily measurable and will obey power-law that good. I think it is a very high risk—very high gain project (very high gain, because if the prediction will be very clear it will be possible to persuade more people that the problem is important).
Let me try to rephrase it in terms of something that can be done in a lab and see if I get your point correctly. We should conduct experiments with humans, identifying what causes sufferings with which intensity, and what happens in the brain during it. Then, if the animal has the same brain regions, it is capable to suffer, otherwise, it is not. But it won’t be the functional approach, we can’t extrapolate it blindly to the AI.
If we want the functional approach, we can only look at the behavior. What we do when we suffer, after it, etc. Then being suffers if it demonstrates the same behavior. Here the problem will be how to generalize human behavior to animals and AI.
Hmmm, but I am not saying that the benevolent simulators hypothesis is false and that I just choose to believe in it because it brings a positive effect. Rather opposite—I think that benevolent simulators are highly likely (more than 50% chance). So it is not a method “to believe in things which are known to be false”. It is rather an argument why they are likely to be true (of course, I may be wrong somewhere in this argument, so if you find an error, I will appreciate it).
In general, I don’t think people here want to believe false things.
You are making a good point. Indeed, the system that would reward authors and experts will be quite complicated, so I was thinking about it on a purely volunteering basis (so in the initial stages it is non-profit). Then, if the group of people willing to work on the project was formed, they may turn it into a business project. If the initial author of the idea is in the project, he may get something, otherwise, no—the idea is already donated, no donations back. I will make an update to the initial post to clarify this point.
As to your idea, I am totally not an expert in this field. Hopefully, we will find the experts for all our ideas (I also have a couple).
Are there any other examples when rationality guides you faster than the scientific approach? If so it would be good to collect and mention them. If no I am pretty suspicious about QM one as well.
First of all, it is my mistake—in the paper they used pain more like a synonym to suffering. They wanted to clarify that the animal avoids tissue damage (heat, punching, electric shock etc.) not just on the place, but learns to avoid it. To avoid it right there is simply nociception that can be seen in many low-level animals.
I don’t know much about the examples you mentioned. For example, bacterias certainly can’t learn to avoid stimuli associated with something bad for them. (Well, they can on the scale of evolution, but not as a single bacteria).
If it is, does it mean that we should all artificial neural network training consider as animal experiments? Should we put something like “code welfare is also animal welfare”?
It is worrisome indeed. I would say, it definitely does not help and only increases a risk. However, I don’t think this country-that-must-not-be-named would start the nuclear war first, simply because it has too much to lose and its non-nuclear opportunities are excellent. This may change in future—so yes, there is some probability as well.
That is exactly the problem. Suppose the Plutonia government sincerely believes, that as soon as other countries will be protected, they will help people of Plutonia to overthrow the government? And they kind of have reasons for such belief. Then (in their model of the world) the world protected from them is a deadly threat, basically capital punishment. The nuclear war, however, is horrible, but there are bomb shelters where they can survive and have enough food inside just for themselves to live till natural death.
The problem is that retaliation is not immediate (missiles takes few hours to reach the goal). For example, Plutonia can demonstratively destroy one object and declare that any attempt of retaliation will be retaliated in double. As soon as other country launches N missiles, Plutonia launches 2 N.
Yes, absolutely, it is the underlying thesis.
Well, “democratic transition” will not necessarily solve that (like basically it did not completely resolve the problem with the end of the Cold War), you are right, so actually, the probability must be higher than I estimated—even worse news.
Is there any other options for decreasing the risk?
From a Russian perspective. Well, I didn’t discuss it with officials in the government, only with the friends who support the current government. So I can only say what they think and feel, and of course, it is just anecdotal evidence. When I explicitly discussed with one of them the possibility of the nuclear war, he stated that this possibility is small and as long as the escalation will be beneficial for Russia he will support it.
I don’t want to go here into politics and discuss what type of government would be better for Russia. I was more interested to estimate the probability of the nuclear war (or other catastrophes mentioned on the main post).
When I say use, I mean actually detonating—not necessarily destroying a big city, but initially maybe just something small.
Within the territory is possible, though I think outside is more realistic (I think the army will be eventually to weak to fight the external enemies with modern technology, but will always be able to fight unarmed citizens).
How should we deal with the cases when epistemic rationality contradicts instrumental? For example, we may want to use placebo effect because one of our values is that healthy is better than sick, and less pain is better than more pain. But placebo effect is based on the fact that we believe pill to be a working medicine that is wrong. Is there any way to satisfy both epistemic and instrumental rationality?