Davidmanheim
Karma: 608
NewTop
I’m definitely not the best person to explain this, since I’m more on the epidemiology side. I understand the molecular clock analyses a bit, and they involve mutation rates plus tracking mutations in different variants, and figuring out how long it should take for the various samples collected at different times to have diverged, and what their common ancestors are.
Thank you! This is a point I keep trying to make, less eloquently, in both bioethics and in AI safety.
We need fewer talking heads making suggestions for how to regulate, and more input from actual experts, and more informed advice going to decision makers. If “professional ethicists” have any role, it should be elicitation, attempting to reconcile or delineate different opinions, and translation of ethical opinions of experts into norms and policies.
I have several short comments about part 3, short not because there is little to say, but because I want to make the points and do not have time to discuss them in depth right now.
1) If multi-agent systems are more likely to succeed in achieving GAI, we should shut up about why they are important. I’m concerned about unilateralist curse, and would ask that someone from MIRI weigh in on this.
2) I agree that multi-agent systems are critical, but for different (non-contradictory) reasons—I think multi-agent systems are likely to be less safe and harder to understand. See draft of my forthcoming article here: https://arxiv.org/abs/1810.10862
3) If this is deemed to be important, the technical research directions point to here are under-specified and too vague to be carried out. I think concretizing them would be useful. (I’d love to chat about this, as I have ideas in this vein. If you are interested in talking, feel free to be in touch—about.me/davidmanheim .)
There is genetic evidence discussed in Hopkins’ “Princes and Peasants: Smallpox in History,” which implies ancient existence of variola viruses, as you note from the Wiki article. The newer paper overstates the case in typical academic fashion in order to sound as noteworthy as possible. The issue with saying that earlier emergence is not the “current” disease of smallpox is that we expect significant evolution to occur once there is sufficient population density, and more once there is selection pressure due to vaccination, and so it is very unsurprising that there are more recent changes. (I discuss this in my most recent paper, https://www.liebertpub.com/doi/pdf/10.1089/hs.2018.0039 )
It’s very clear that a precursor disease existed in humans for quite a while. It’s also very clear that these outbreaks in thin populations would have continued spreading, so I’m unconvinced that the supposed evidence of lack due to Hippocrate’s omission, and the lack of discussion in the old and new testament is meaningful. And regarding the old testament, at least, the books aren’t great with describing “plagues” in detail, and there are plenty of times we hear about some unspecified type of plague or malady as divine punishment.
So the answer depends on definitions. It’s unclear that there is anything like a smallpox epidemic as the disease currently occurs in a population that is not concentrated enough for significant person-to-person spread. If that’s required, we have no really ancient diseases, because we defined them away.
The model implies that if funding and prestige increased, this limitation would be reduced. And I would think we don’t need prestige nearly as much as funding—even if near-top scientists were recruited and paid the way second and third string major league players in most professional sports were paid, we’d see a significant relaxation of the constraint.
Instead, the uniform wage for most professors means that even the very top people benefit from supplementing their pay with consulting, running companies on the side, giving popular lectures for money, etc. - all of which compete for time with their research.
Yes, this might help somewhat, but there is an overhead / deduplication tradeoff that is unavoidable.
I discussed these dynamics in detail (i.e. at great length) on Ribbonfarm here.
The large team benefit would explain why most innovation happens near hubs / at the leading edge companies and universities, but that is explained by the other theories as well.
The problem with fracturing is that you lose coordination and increase duplication.
I have a more general piece that discusses scaling costs and structure for companies that I think applies here as well—https://www.ribbonfarm.com/2016/03/17/go-corporate-or-go-home/
This seems to omit a critical and expected limitation as a process scales up in the number of people involved—communication and coordination overhead.
If there is low hanging fruit, but everyone is reaching for it simultaneously, then doubling the number of researchers won’t increase the progress more than very marginally. (People with slightly different capabilities implies that the expected time to success will be the minimum of different people.) But even that will be overwhelmed by the asymptotic costs for everyone to find out that the low-hanging fruit they are looking for has been picked!
Is there a reason not to think that this dynamic is enough to explain the observed slowdown—even without assuming hypothesis 3, of no more low-hanging fruit?
The paper is now live on Arxiv: https://arxiv.org/abs/1811.09246
In part, I think the implication of zero-sum versus non-zero sum status is critical. Non-zero sum status is “I’m the best left-handed minor league pitcher by allowed runs” while zero-sum status is “by total wealth/power, I’m 1,352,235,363rd in the world.” Saying we only have on positional value for status seemingly assumes the zero-sum model.
The ability to admit these non-zero sum status signals has huge implications for whether we can fulfill values. If people can mostly find relatively high-position niches, the room for selection on noise and path-dependent value grows.
This also relates to TAG’s point about whether we care about “value” or “moral value”—and I’d suggest there might be moral value in fulfilling preferences only if they are not zero-sum positional ones.
This is a good point, but I’ll lay out the argument against it.
To start, I’m personally skeptical of the claim that preferences and moral values can be clearly distinguished, especially given the variety of value systems that people have preferred over time, or even today.
Even if this is false, we seem to see the same phenomenon occur with moral values. I think the example of obvious differences in the relative preference for saving dogs, the elderly, or criminals points to actual differences in values—but as I argued above, I think this is a heavily optimized subspace of a moral intuition towards liking life which is now largely selecting on noise. But the difference in moral conclusions that follow from assigning animal lives exactly zero versus smaller-than human but nonzero value are huge.
Yes, and that’s closely related to the point I made about ” we’re adaptation executioners, not fitness maximizers.”
My point is a step further, I think—I’m asking what decides which things we plan to do? It’s obviously our “preferences,” but if we’ve already destroyed everything blue, the next priority is very underspecified.
I agree that positional goods are important even in the extreme, but:
1) I don’t think that sexual desires or food preferences fit in this mold.
2) I don’t think that which things are selected as positional goods (perhaps other than wealth and political power) is dictated by anything other than noise and path dependence—the best tennis player, the best DOTA player, or the most cited researcher are all positional goods, and all can absorb arbitrary levels of effort, but the form they take and the relative prestige they get is based on noise.
Values Weren’t Complex, Once.
Yes, this very much resonates with me, especially because a parallel issue exists in biosecurity, where we don’t want to talk publicly about how to work to prevent things that we’re worried about because it could prompt bad actors to look into those things.
The issues here are different, but the need to have walls between what you think about and what you discuss imposes a real cost.
Oversight of Unsafe Systems via Dynamic Safety Envelopes
I don’t think humans have collaboration as a default—it’s only because evolution was due to social pressure that this occurs at all, and it occurs primarily at the social-structure level, not as an outcome of individual effort.
Even if this is wrong, however, non-GAI systems can pose existential risks.
I’m stuck part-way through on #4 - I assume there is a way to do this without the exhaustive search I’m running into needing.
I’m going to try (nested) induction. Define triangles by side size, measured in nodes.
Induction base step: For n=2, there must be exactly one trichromatic edge.
Induction step: If there are an odd number of tri-chromatic edges for all triangles n=x, we must show that this implies the same for n=x+1.
We create all possible new triangles by adding x+1 nodes on one of the sides, then allow any of the previous x nodes on that side to change. Without loss of generality, assume we add x+1 edges to the bottom (non-red) side. These must be green or blue. The previous layer can now change any number of node-colors. We now must prove this by induction on color changes of nodes in the second-to-bottom layer to be red. (If they flip color otherwise, it is covered by a different base case.)
First, base step, assume no nodes change color. Because the previous triangle had an odd number of trichromatic edges, and the new edge is only green+blue, no new trichromatic edges were created.
Induction step: There is an x+1 triangle with an odd number of trichromatic vertices, and one node in the second-to-bottom layer changes to red. This can only create a new tri-cromatic triangle in one of the six adjacent triangles. We split this into (lots of) cases, and handle them one at a time.(Now I get into WAY too many cases. I started and did most of the edge-node case, but it’s a huge pain. Is there some other way to do this, presumably using some nifty graph theory I don’t know, or will I need to list these out? Or should I not be using the nested induction step?)
Pointers welcome!
I am having trouble figuring out why #2 needs / benefits from Sperner’s Lemma.
But I keep going back to the proof that I’m comfortable with, which depends on connectedness, so I’m clearly missing an obvious alternative proof that doesn’t need topology.
Yes.
But to clarify, I don’t think the Antonine plague is quite the same as modern ones, for the simple reason that it could only spread over a fairly limited geographic region, and it could not become endemic because of population density constraints. Smallpox evolution is driven by selection pressure in humans, and the “500 years old” claim is about that evolution, not about whether it affected humans at any time in the past. That said, it absolutely matters, because if the original source of smallpox was only 500 years ago, where did it come from?
The question is how smallpox evolved, and what variant was present prior to the 1500s. It’s plausible that Horsepox, which was probably the source for the vaccine strain, or Cowpox, spread via intermediate infections in cats, were the source—but these are phylogenetically distant enough that, from my limited understanding, it’s clearly implausible that it first infected humans and turned into modern smallpox at recently as the 1500s. (But perhaps this is exactly the claim of the paper. I’m unclear.) Instead, my understanding is that there must have been some other conduit, and it seems very likely that it’s related to a historically much earlier human pox virus—thousands of years, not hundreds.