Eliezer’s recent kidding not kidding death with dignity post suggests that our chances of survival are so low that we should just focus on going out with some semblance of dignity (ie. “at least we made an attempt that wasn’t truly pathetic).
Or at least that’s what it seems to be claiming initially. If you read it to the end it becomes clear that Eliezer is proposing a frame for thinking about how to act in scenarios with a low probability of success[1]. In particular, he seems to be criticizing the tendency of people to say “well we need to assume X because otherwise we would be doomed anyway” as the tendency is to power on multiple assumptions and end up focusing on and over specific-case. In contrast, Eliezer suggests that it is better to position yourself sucks that you would be able to leverage positive moral violations in general as then you aren’t just betting on one particular scenario.
I often find that reframing a problem is quite conducive to producing solutions, so I’m asking how we could actually die with more dignity[2].
The way to die with dignity is to genuinely intend to succeed even as we accept that we will likely fail.
Research how to transfer knowledge from trained ML systems to humans.
An example: It was a great achievement when AlphaGo and later systems defeated human go masters. It would be an even greater achievement for the best computer go systems to lose to human go masters—because that would mean that the knowledge these systems had learned from enormous amounts of self-play had been successfully transferred to humans.
Another example: Machine learning systems that interpret medical X-ray images or perform other diagnostic functions may become better than human doctors at this (or even if not better overall, better in some respects). Transferring their knowledge to human doctors would produce superior results, because the human doctor could integrate this knowledge with other knowledge that may not available to the computer system (such as the patient’s demeanor).
From the x-risk standpoint, it seems quite plausible that a better ability to transfer knowledge would both allow humans to more successfully “keep up” with the AIs, and to better understand how they may be going wrong.
This line of research has numerous practical applications, and hence may be feasible to promote, especially with a bit of “subsidy” from those concerned about x-risks. (Without a subsidy, it’s possible that just enhancing the capability of ML systems would seem like the higher-return investment.)
This somewhat happenned in chess : today’s top players are much stronger than twenty years ago, mainly thanks to new understanding brought by the computers. Carlsen or Caruana would probably beat Deep Blue handily.
One method would be to take advantage of low-hanging fruit not directly related to X-risk. Clearly motivation isn’t enough to solve these problems (and I’m not just talking about alignment), so we should be trying to optimize all our resources, and that includes getting rid of major bottlenecks like [the imagined example of] hunger killing intelligent, benevolent potential-researchers in particular areas because of a badly-designed shipping route.
A real-life example of this would be the efforts of the Rationalist community to promote more efficient methods of non-scientific analysis (i.e. cases where you don’t have the effort required for scientific findings, but want a right answer anyway). This helps not only in X-risk efforts, but also in the preliminary stages of academic research, and [presumably] entrepreneurship as well. We could step up our efforts in this, particularly in college environments where it would influence people’s effectiveness whether or not they bought into other aspects of this subgroup’s culture like the urgency of anti-X-risk measures.
Another aspect is to diverge in multiple different directions. We’re essentially searching for a miracle at this point (to my understanding, in the Death with Dignity post Eliezer’s main reason to reject unethical behaviors that might, maybe, possibly lead to success is that they’re still less reliable than miracles and reduce our chances of finding any). So we need a much broader range of approaches to solving or avoiding these problems, to increase the likelihood that we get close enough to a miracle solution to spot it.
For instance, most effort on AGI safety so far has focused on the alignment and control problems, but we might want to put more attention to how we might keep up with a self-optimizing AGI by augmenting ourselves, so that human society was never dominated by an inhuman (and thus likely unaligned) cognition. This would involve both the existing line of study in Intelligence Augmentation (IA), but also ways to integrate it with AI insights to keep ahead of an AI in its likely fields of superiority, and also relates to the social landscape of AI in that we’d need to draw resources and progress away from autonomous AI and towards IA.
Working on global poverty seems unlikely to be a way of increasing our chances of succeeding at alignment. If anything, this would likely increase both the number of future alignment and capacity researchers. So it’s unlikely to significantly increase our chances.
Augmentation is potentially more promising. I guess my main worry is that if we plug computers into our brains, then this makes us more vulnerable to hacking and so it might even make it easier for things to go wrong. That said, it could still be positive in expectation.
“Working on global poverty seems unlikely to be a way of increasing our chances of succeeding at alignment. If anything, this would likely increase both the number of future alignment and capacity researchers. So it’s unlikely to significantly increase our chances.”
A fair point regarding alignment (I hadn’t thought about how it would affect AI researchers as well), but I was more thinking from the perspective of X-risk in general.
AI alignment is one issue that doesn’t seem to be significantly affected either way by this, but we also have things like alignment of organizations towards public interest (which is currently a fragile, kludged-together combination of laws and occasional consumer/citizenry strikes) or the increasing rate of natural disasters like pandemics and hurricanes (which requires both technical and social aspects for a valid solution), and both of these have the potential to lead to at least civilizational collapse, if not human extinction (as examples, through “large-scale nuclear war for the sake of national sovereignty” and “lack of natural resources or defense against natural disasters”, respectively).
It seems to me that it’s still in question whether AI alignment (or more generally, ethical/safety controls on impending technological advancements) is the earliest X-risk in our way, and having a more varied set of workers on these problems would be helpful for ensuring we survive many of the others while (as you mentioned) not significantly affecting the balance of this particular problem one way or another.
Even so, it seems much harder to attempt to influence any of these through economic development than through anything more direct. Like if we increased yearly economic growth by 5% (for example 2% to 2.1%), what effect would you expect that to have? Given how big the world is, is it really easier to increase the number of x-risk researchers by increasing the economic growth of the entire world, rather than just engaging in standard movement building?
I suspect the impact is net-negative because increasing both amounts of researchers shortens the timelines and longer timelines increase our odds as EA and AI safety are becoming much more established.
“Like if we increased yearly economic growth by 5% (for example 2% to 2.1%), what effect would you expect that to have?”
From my personal experience, academics have a tendency and preference to work on superficially-beneficial problems; Manhattan Projects and AI alignment groups both exist (detrimental and non-obviously beneficial, respectively), but for the most part we have projects like eco-friendly technology and efficient resource allocation in specified domains.
Due to this, greater economic growth means more resources to bring to bear for other scientific/engineering problems, due to research on superficially-beneficial subjects like power-generation, efficiency, quantum computing, etc. As noted in my previous comment, the economic growth (and these increased resources as well) will also lead to an increased number of researchers and engineers.
Fields of study considered as X-risks are often popular enough that development to dangerous levels is actually an urgent possibility. As such, I would expect them to be bounded by academic development rather than resource availability (increased hardware capabilities might be a bottleneck for AGI development, but at this point I doubt it, as at least one [not-vetted-by-me] analysis I’ve encountered suggests (assuming perfectly-efficient computation using parallel graph-based operations) that modern supercomputers are only 1 or 2 orders of magnitude away from the raw computational ability of the human brain).
(Increased personnel is beneficial to these fields, but that’s addressed below and in the second part of this comment.)
So the changes caused by these increased resources would mostly occur in other fields, which are generally geared towards either increased life/quality-of-life (which encourages less ‘practical’ pursuits like philosophy and unusual worldviews (e.g. Effective Altruism), potentially increasing deviation from the economic incentives promoting dangerous technology, and also feeds back into economic growth) or better general understanding of the world (which accelerates dangerous, non-dangerous, and anti-X-risk (e.g. alignment) research to a similar degree).
Regarding that second category, many conventional fields are actually working directly on possible solutions to X-risk problems, whether or not they believe in the dangers. Climate change, resource shortages, and asteroid risk are all partly addressed by space research, and the first two are also relevant to ecological research. Progress in fields like psychology/neurology & sociology/game-theory is potentially applicable to AI alignment, and can also be used to help encourage large-scale coordination between organizations. The benefits from these partially counterbalance what impact the economic growth does have on more dangerous fields like directed AGI research.
And on a separate note, I would consider “dying with dignity” to also mean “not giving up on improving people’s lives just because we’re eventually all going to die”. This is likely not what Eliezer meant in his post, but I doubt he (or most people) would be actively opposed to the idea. From this perspective, many conventional research directions (which economic growth tends to help) are useful for dying with dignity, even the ones that don’t directly apply to X-risk.
“I suspect the impact is net-negative because increasing both amounts of researchers shortens the timelines and longer timelines increase our odds as EA and AI safety are becoming much more established.”
This is going into more speculative territory, since I doubt either of us are experienced professional sociologists. Still, to my knowledge paradigm-changes in a field are rarely a result of convincing the current members of an issue; they usually involve new entrants, without predefined biases and frameworks, leaning towards the new way of looking at things.
So the rate of EA & AI safety becoming established would also increase significantly if there was a large influx of new academics with an interest in altruistic academic efforts (since their communities were helped by such efforts), meaning the increase in research population should be more balanced towards safety/alignment than the current population is.
Whether this change in proportion is sufficiently unbalanced to counteract the changes in progress of technologies like AGI is difficult to judge.
For one thing, due to threshold effects I’d expect research progress vs research population to be something like an irregular step-function with sigmoid-shaped inter-step transitions on either the base level or one of the lower-level differentials, meaning population doesn’t have a direct relation to progress levels.
For another, as you mentioned, other talented individuals in this influx would be pushed towards these fields because of the challenges and income they offer, and while this seems at first glance to be the weaker of the two incentives, it may well be the greater and thus falsify my assumption that EA/alignment would come out better in population growth.
In a surface-level analysis like this I generally assume equivalence in the important aspects (research progress, in this case) for such ambiguous situations, but you are correct that it might be weighted towards the less-desirable outcome.
I don’t know how much money has been spent on AI safety, but if we go out without having spent $1 billion, then that would seem undignified. Same if we went out without spending at least 10% of our available funds.
We could, in principle, decide that survival of humanity in current form (being various shades of unlikely depending on who you believe), is no longer a priority and focus on different goals what are still desirable in the face of likely extinction. For example:
See if any credible MAD schemes are possible when AGI is one of the players
Accept survival in a reduced capacity, i.e. kept as a pet or a battle-tested biological backup
Ensuring that AGI which kills us can at least do something interesting later, i.e. it’s something smarter than a fixed-goal paperclip optimizer
Preemptively stopping any unambiguously hostile activities towards the future AGI like alignment research and start working on alignment of human interests towards AGI’s instead
These are just off from the top of my head and I’m sure there are many more available once survival requirement is removed
Alignment research is not necessarily hostile towards AGIs. AGI also has to solve alignment to cooperate with each other and not destroy everything on earth.
I’m not sure about this as mere limitation of AGI capability (to exclude destruction of humanity) is, in a sense, a hostile act. Control of AGI as in AI control problem certainly is hostile
The Fermi paradox does suggest that multiple AGIs that don’t solve the control problem would also self-destruct.
Why can’t one of the AGIs win? Fermi paradox potentially has other solutions as well
It’s possible to have an AGI war and one AGI wins and then decides to stop duplicating itself but generally it’s likely that AGIs that do duplicate themselves are more powerful then those that don’t because self duplication is useful.
Fix Swapcard’s user interface before the apocalypse.