Yes, good point that I hadn’t thought of, thanks. It’s very easy to imagine far-future technology in one respect and forget about it entirely in another.
To rescue my scenario a little, there’ll be an energy cost in transporting the iron together; the cheapest way is to move it very slowly. So maybe there’ll be paperclips left for a period of time between the first pass of the harvesters and the matter ending up at the local black hole harvester.
That, and/or increased sweating, and/or larger temperature gain between inspired and expired air, or wearing fewer/thinner clothes. There’s lots of ways to dump heat.
I would definitely expect someone with a faster metabolism to put out more total net heat, which is measurable with difficulty, and also consume oxygen faster (and produce carbon dioxide faster) which is measurable with some difficulty, but a lot less.
Therefore they must not vary all that much in terms of metabolism.
I don’t think that follows, or at least not without a lot of other explanation, even if you grant that temperature doesn’t vary in any significant way between people (which I’m not sure I do). The body has multiple mechanisms for maintaining temperature, of which metabolic rate is only one. It seems entirely plausible to me that people run their metabolisms at different rates and adjust their peripheral vasodilation and sweating rate to balance it all out near 37 C/98 F. Core temperature might vary between people by only a few degrees, but surface temperature varies much more widely.
Also, they were not just AIDS researchers but AIDS activists and campaigners. The conference they were going to was expecting 12-15,000 delegates (depending on the report); it’s the most prominent international conference in the area but far from the only one. As you say, a terrible loss, particularly for those close to the dead. The wider HIV/AIDS community will be sobered, but it will not be sunk. If nothing else, they coped with far higher annual death rates before effective therapies became widespread in the developed world.
The story of this story does helpfully remind us that the other ‘facts’ about this situation—which we know from the same media sources—may be similarly mistaken.
Much of modern medicine involves covering up symptoms with drugs proven to do this, without understanding the underlying cause of the symptom.
What, really? There certainly is a lot of that approach around, but it’s not what I think of when I think of modern medicine, as opposed to more traditional forms. Can you give examples?
Most of the ones I can think of are things that have fallen to the modern turn to evidence-based practice. The poster-child one in my head is the story of H. pylori and how a better understanding of the causes of gastritis and gastric ulcers has led to better treatments than the old symptom-relieving approaches. (And I’ll tell you what, although Zantac/Ranitidine is only a symptomatic reliever, it was designed to do that job based on a thorough understanding of how that symptom comes about, and it’s bloody good at it, as anyone who’s had it for bad heartburn or reflux can attest.)
When I think of modern medicine, I think of things like Rituximab, which is a monoclonal antibody designed with a very sophisticated understanding of how the body’s immune system works—it targets B cells specifically, and has revolutionised drug treatment for diseases like non-Hodgkin’s lymphomas where you want to get rid of B cells. So much so that for some of those lymphomas, we don’t have very robust 5 year survival data, because the improvement over traditional chemotherapy alone is so large that the old survival data is no use (we know people will live much longer than that), and Rituximab hasn’t been widely used for long enough to get new data. In the last 25 years our understanding of cancer has gone from “it’s mutations in the genes, probably these ones” to vast databases of which specific mutations at which specific locations on which specific genes are associated with which specific cancer symptoms, and how those are correlated with prognosis and treatment. And as a result cancer survival rates have improved markedly. We don’t have “A Cure For Cancer”, and we now know we never will, any more than we can have “A Cure For Infection”, but we do have a good enough understanding of how it happens to get much better at reducing its impact.
Even modern medical disasters like Vioxx are hardly a result of a lack of understanding the underlying cause, but more us learning more about other complexities of human biology. Admittedly we don’t yet fully understand how pain works, but we do know enough to know that targeting COX-2 exclusively (rather than COX-1 as well, which looks after your gut lining) would be safer for your gut. This is understanding down at the molecular level. It turns out in large scale studies that they are safer for your gut, but of course they’re not very safe for your heart, so we’ve stopped using them. And actually doing the full-scale research on modern rationally-designed drugs like Vioxx suggests that similar old drugs (that we never bothered to test) have the same effect on hearts.
Interesting stuff, thanks; looking forward to the rest of the series.
As an aside, this makes the benefits of being able to rely on trust most of the time very apparent. Jack and Jill can coordinate very simply and quickly if they trust each other to honestly disclose their true value for the project. They don’t even need to be able to trust 100%, just trust enough that on average they lose no more to dishonesty than the costs of more complex and sophisticated methods of bargaining. (Which require more calculating capacity than unaided humans have evolved.)
I find similar techniques help with my children.
It seems closely related to the technique where, to stop them doing something you don’t want them to do, you encourage them to do something else that prevents them from doing the first thing. (There’s a snappy name for this that I’ve forgotten.) So, for example, stopping them from bothering another child by getting them interested in an entirely different activity.
I really don’t think we have to posit nanoassemblers for this particular scenario to work. Robot drones are needed, but I think they fall out as a consequence of currently existing robots and the all-singing all-dancing AI we’ve imagined in the first place. There are shedloads of robots around at the moment—the OP mentioned the existence of Internet-connected robot-controlled cars, but there are plenty of others, including most high tech manufacturer. Sure, those robots aren’t autonomous, but they don’t need to be if we’ve assumed an all-singing all-dancing AI in the first place. I think that might be enough to keep the power and comms on in a few select areas with a bit of careful planning.
Rebuilding/restarting enough infrastructure to be able to make new and better CPUs (and new and better robot extensions of the AI) would take an awfully long time, granted, but the AI is free of human threat at that point.
One thing I should mention where I wasn’t able to get a very good match between my own observations and mainstream science.
The Sun and the Moon are very, very close in their apparent diameter in the sky. They are almost exactly the same size. You can measure them yourself and compare, although this is a bit fiddly; I certainly got well within my own measurement errors, although those errors were large. However, you can verify it very easily and directly at the time of solar eclipses. They are so near in size that the wobbliness of the Moon’s orbit means that sometimes the Sun is just-smaller than the Moon (when you get a total eclipse) and sometimes it is just-bigger (when you get an annular eclipse).
But they are very, very different in their actual size, and in their distance from the Earth. In Father Ted terms, the Moon is small and close; the Sun is large and far away. In rough terms, the Moon is 400,000 km away and 3,400 km across, and the Sun is 150m km away and 1.4m km across. You don’t have to change any one of those four measurements much for them to be quite different apparent sizes from the Earth. Indeed, if you do the calculations (which I can personally attest to), if you go back far enough in time they weren’t the same apparent size, and nor are they if you go forward a long way in to the future.
Why? Why this coincidence? And why is it only happening at just the times when humans are around to observe it?
So far as I know, we have no good theories apart from “it just happened to work out that way”. This is pretty unsatisfying.
An AI controlling a company like Google would be able to, say, buy up many of the world’s battle robot manufacturers, or invest a lot of money into human-focused bioengineering), despite those activities being almost entirely unrelated to their core business, and without giving any specific idea of why.
Indeed, on the evidence of the press coverage of Google’s investments, it seems likely that many people would spend a lot of effort inventing plausible cover stories for the AI.
I’ll grant that “a very large proportion of the world’s computing resources” was under-specified and over-stated. Sorry.
Bedford Level Experiment [...] has the disadvantage that it shows that the Earth is flat.
I love this. As it happens, I live quite near Bedford and am terribly tempted to actually try it one day. (Edit Looking closer, turns out the Bedford Level is in Norfolk, not Bedfordshire, so a little less nearby than I thought.)
There are loads of fun ways of verifying that the Earth isn’t flat. Some of these were easily available to the ancients—e.g. the shape of the shadow of the Earth on the Moon during a lunar eclipse (it’s always a curve). Others are easier now than they used to be—e.g. the variations in the constellations you can see as you travel north-south (it’s much easier to travel far enough to see this than it used to be).
Some, however, simply weren’t available.
My favourite explanation for how we know for sure the Earth is round is that we’ve been up in to space and looked. You can even verify this yourself with a GoPro and a high-altitude balloon, which many hobbyists have done.
I spent quite a lot of time many years ago doing my own independent checks on astronomy.
I started down this line after an argument with a friend who believed in astrology. It became apparent that they were talking about planets being in different constellations to the ones I’d seen them in. I forget the details of their particular brand of astrology, but they had an algorithm for calculating a sort-of ‘logical’ position of the planets in the 12 zodiacal signs, and this algorithm did not match observation, even given that the zodiacal signs do not line up neatly with modern constellations. They were scornful that I was unable to tell them where, say, Venus would be in 12 years time, or where it was when I was born.
So challenged, I set to.
The scientific algorithms for doing this are not entirely trivial. I got hold of a copy of Jean Meeus’ Astronomical Algorithms, and it took me quite a lot of work to understand them, and then even longer to implement them so I could answer that sort of question. They are hopelessly and messily empirical (which I take as a good sign) - there is a daunting number of coefficients. Eventually I got it working, and could match observation to prediction of planetary positions to my satisfaction—when I looked at them, the planets were where my calculations said they should be, more or less.
It’s hard with amateur equipment to measure accurate locations in the sky (e.g. how high and in which direction is a particular star at a particular time), but relative ones are much easier (e.g. how close is Venus to a particular star at a particular time). The gold standard for this sort of stuff is occultations—where you predict that a planet will occult (pass in front of) a star. There weren’t any of those happening around the time I was doing it, but I was able to verify the calculations for other occultations that people had observed (and photographed) at the date and times I had calculated.
These days, software to calculate this stuff—and to visualise it, which I never managed—is widely available. There are many smartphone apps that will show you these calculations overlaid on to the sky when you hold your phone up to it. (Although IME their absolute accuracy isn’t brilliant, which I think is due to the orientation sensors being not that good.) This makes checking these sorts of predictions very, very easy. Although of course you can’t check that there isn’t, say, a team of astronomers making observations and regularly adjusting the data that gets to your phone.
I was also able to independently replicate enough of Fred Espenak’s NASA eclipse calculations to completely convince me he was right. (After I found several bugs in my own code.) Perhaps the most spectacular verification was replicating the calculations for the solar eclipse of 11 August 1999. I was also able to travel to the path of totality in France, and it turned up slap on time and in place. This was amazing, and I strongly urge anyone reading this to make the effort to travel to the path of totality of any eclipse they can.
Until I’d played around with these calculations, I hadn’t appreciated just how spectacularly accurate they have to be. You only need a teeny-tiny error in the locations of the Sun/Moon/Earth system for the shadow cast by the moon on the Earth to be in a very different place.
I also replicated the calculations for the transit of Venus in 2004. I was able to observe it, and it took place exactly as predicted so far as I was able to measure—to within, say, 10 seconds or so. (I didn’t replicate the calculations for the transit in 2012 - no time and I’d forgotten about how my ghastly mess of code worked—and I wasn’t able to observe it either, since it was cloudy where I was at the time.)
More recently, you can calculate Iridium flares and ISS transits. Again, you have to be extremely accurate in calculations to be able to predict where they will occur, and they turn up as promised (except when it’s cloudy). And again, there are plenty of websites and apps that will do the calculations for you. With a pair of high-magnification binoculars you can even see that the ISS isn’t round.
All this isn’t complete and perfect verification. But it’s pretty good Bayesian evidence in the direction that all that stuff about orbits and satellites is true.
If you are at all mathematical, you can verify that relativity affects GPS signals by calculating what difference both special relativity (satellite clock moving faster than clock on Earth, hence slower) and general relativity (satellite clock higher up the gravitational field than clock on Earth) would make to timekeeping and hence accuracy of location. The effects work against each other, but one is larger than the other.
You can verify accuracy of location of a GPS yourself. IME this is almost always considerably less accurate than published estimates by the device manufacturer, but still impressive. However, you need to be careful—most smartphones use multiple technologies to determine their location, not just GPS, so will be more accurate than the GPS signal can possibly be.
To be fair to the medieval, their theories about how one can build large, beautiful buildings were pretty sound.
Do you believe that if Obama were to ask the NSA to take over Russia, that the NSA could easily do so?
No. I think the phrase “take over” is describing two very different scenarios if we compare “Obama trying to take over the world” and “a hypothetical hostile AI trying to take over the world”. Obama has many human scruples and cares a lot about continued human survival, and specifically not just about the continued existence of the people of the USA but that they thrive. (Thankfully!)
I entirely agree that killing huge numbers of people would be a stupid thing for the actual NSA and/or Obama to do. Killing all the people, themselves included, would not only fail to achieve any of their goals but thwart (almost) all of them permanently. I was treating it as part of the premises of the discussion that the AI is at least indifferent to doing so: it needs only enough infrastructure left for it to continue to exist and be able to rebuild under its own total control.
a long-term plan. But those are very risky. The longer it takes, the higher the chance that your plan is revealed.
Yes, indeed, the longer it takes the higher the chance that the plan is revealed. But a different plan may take longer but still have a lower overall chance of failure if its risk of discovery per unit time is substantially lower. Depending on the circumstances, one can imagine an AI calculating that its best interests lie in a plan that takes a very long time but has a very low risk of discovery before success. We need not impute impatience or hyperbolic discounting to the AI.
But here I’ll grant we are well adrift in to groundless and fruitless speculation: we don’t and can’t have anything like the information needed to guess at what strategy would look best.
Anyway, the assumption that an AI could understand human motivation, and become a skilled manipulator, is already too far-fetched for me to take seriously.
I wouldn’t say I’m taking the idea seriously either—more taking it for a ride. I share much of your skepticism here. I don’t think we can say that it’s impossible to make an AI with advanced social intelligence, but I think we can say that it is very unlikely to be achievable in the near to medium term.
This is a separate question from the one asked in the OP, though.
Could the NSA, the security agency of the most powerful country on Earth, implement any of these schemes?
Er, yes, very easily.
Gaining effective control of the NSA would be one route to the AI taking over. Through, for example, subtle man-in-the-middle attacks on communications and records to change the scope of projects over time, steathily inserting its own code, subtle manipulation of individuals, or even straight-up bribery or blackmail. The David Petraeus incident suggests op sec practice at the highest levels is surprisingly weak. (He had an illicit affair when he was Director of the CIA, which was stumbled on by the FBI in the course of a different investigation as a result of his insecure email practices.)
We’ve fairly-recently found out that the NSA was carrying out a massive operation that very few outsiders even suspected—including most specialists in the field—and that very many consider to be actively hostile to the interests of humanity in general. It involved deploying vast quantities of computing resources and hijacking those of almost all other large owners of computing resources. I don’t for a moment believe that this was an AI takeover plan, but it proves that such an operation is possible.
That the NSA has the capability to carry out such a task (though, mercifully, not the motivation) seems obvious to me. For instance, some of the examples posted elsewhere in the comments to this post could easily be carried out by the NSA if it wanted to. But I’m guessing it seems obvious to you that it does not have this capability, or you wouldn’t have asked this question. So I’ve reduced my estimate of how obvious this is significantly, and marginally reduced my confidence in the base belief.
Alas, I’m not sure we can get much further in resolving the disagreement without getting specific about precise and detailed example scenarios, which I am very reluctant to do, for the reasons mentioned above. any many besides. (It hardly lives up to the standards of responsible disclosure of vulnerabilities.)
your hypothetical artificial general intelligence
It’s not mine. :-) I am skeptical of this premise—certainly in the near term.
Another class of routes is for the AI to obtain the resources entirely legitimately, through e.g. running a very successful business where extra intelligence adds significant value. For instance, it’s fun to imagine that Larry Page and Sergey Brin’s first success was not a better search algorithm, but building and/or stumbling on an AI that invented it (and a successful business model) for them; Google now controls a very large proportion of the world’s computing resources. Similarly, if a bit more prosaically, Walmart in the US and Tesco in the UK have grown extremely large, successful businesses based on the smart use of computing resources. For a more directly terrifying scenario, imagine it happening at, say, Lockheed Martin, BAE Systems or Raytheon.
These are not quick, instant takeovers, but I think it is a mistake to imagine that it must happen instantly. An AI that thinks it will be destroyed (or permanently thwarted) if it is discovered would take care to avoid discovery. Scenarios where it can be careful to minimise the risk of discovery until its position is unassailable will look much more appealing than high-risk short-term scenarios with high variance in outcomes. Indeed, it might sensibly seek to build its position in the minds of people-in-general as an invaluable resource for humanity well before its full nature is revealed.
For a fully-capable sophisticated AGI, the question is surely trivial and admits of many, many possible answers.
One obvious class of routes is to simply con the resources it wants out of people. Determined and skilled human attackers can obtain substantial resources illegitimately—through social engineering, fraud, directed hacking attack, and so on. If you grant the premise of an AI that is smarter than humans, the AI will be able to deceive humans much more successfully than the best humans at the job. Think Frank Abagnale crossed with Kevin Mitnick, only better, on top of a massive data-mining exercise.
(I have numerous concrete ideas about how this might be done, but I think it’s unwise to discuss the specifics because those would also be attack scenarios for terrorists, and posting about such topics is likely—or ought to be likely—to attract the attention of those charged with preventing such attacks. I don’t want to distract them from their job, and I particularly don’t want to come to their attention.)
Empirically we seem to be converging on the idea that the expansion of the universe continues forever (see Wikipedia for a summary of the possibilities), but it’s not totally slam-dunk yet. If there is a Big Crunch, then that puts a hard limit on the time available.
If—as we currently believe—that doesn’t happen, then the universe will cool over time, until it gets too cold (=too short of negentropy) to sustain any given process. A superintelligence would obviously see this coming, and have plenty of time to prepare—we’re talking hundreds of trillions of years before star formation ceases. It might be able to switch to lower-power processes to continue in attenuated form, but eventually it’ll run out.
This is, of course, assuming our view of physics is basically right and there aren’t any exotic possibilities like punching a hole through to a new, younger universe.