It’s a recitation of arguments and anecdotes in favor of secrecy, so of course it’s an argument in that direction. If that wasn’t the intention there would also have been anti-secrecy arguments and anecdotes.
ewbrownv
Karma: 328
I don’t actually agree with the assertion, but I can see at least one coherent way to argue it. The thinking would be:
The world is currently very prosperous due to advances in technology that are themselves a result of the interplay between Enlightenment ideals and the particular cultures of Western Europe and America in the 1600-1950 era. Democracy is essentially irrelevant to this process—the same thing would have happened under any moderately sane government, and indeed most of the West was neither democratic nor liberal (in the modern sense) during most of this time period.
The recent outbreak of peace, meanwhile, is due to two factors. Major powers rarely fight because they have nuclear weapons, which makes war insanely risky even for ruling elites. Meanwhile America has become a world-dominating superpower with a vested interest in maintaining the status quo, so many small regional conflicts are suppressed by the threat of American intervention.
That gets us to “democracy/liberalism” doesn’t get credit for making things better. To go from there to “democracy / liberalism makes things worse” you just have to believe that modern liberal societies are oppressive in ways that plausible alternatives wouldn’t be, which is somewhat plausible if your personal values conflict with liberal thinking.
In reality I suspect that the alternative histories mostly involve autocratic governments banning innovation and fighting lots of pointless wars, which is why I don’t buy the argument. But the evidence that liberal democracy is better than, say, a moderately conservative republic or a constitutional monarchy, is actually pretty weak. The problem is the nice alternatives to democracy are rare, because normally a country that starts moving away from autocracy ends up falling completely into the populism attractor instead of stopping somewhere along the way.
Historically it has never worked out that way. When a society gets richer the people eat more and better food, buy more clothes, live in bigger houses, buy cars and appliances, travel more, and so on. Based on the behavior of rich people we can see that a x10 or even x100 increase from current wealth levels due to automation would just continue this trend, with people spending the excess on things like mansions, private jets and a legion of robot servants.
Realistically there’s probably some upper limit to human consumption, but it’s so far above current production levels that we don’t see much hint of where it would be yet. So for most practical purposes we can assume demand is infinite until we actually see the rich start systematically running out of things to spend money on.
Because you can’t create real, 100% physical isolation. At a minimum you’re going to have power lines that breach the walls, and either people moving in and out (while potentially carrying portable electronics) or communication lines going out to terminals that aren’t isolated. Also, this kind of physical facility is very expensive to build, so the more elaborate your plan is the less likely it is to get financed.
Military organizations have been trying to solve these problems ever since the 1950s, with only a modest degree of success. Even paranoid, well-funded organizations with a willingness to shoot people have security breaches on a fairly regular basis.
Indeed. What’s the point of building an AI you’re never going to communicate with?
Also, you can’t build it that way. Programs never work the first time, so at a minimum you’re going to have a long period of time where programmers are coding, testing and debugging various parts of the AI. As it nears completion that’s going to involve a great deal of unsupervised interaction with a partially-functional AI, because without interaction you can’t tell if it works.
So what are you going to do? Wait until the AI is feature-complete on day X, and then box it? Do you really think the AI was safe on day X-1, when it just had a couple of little bugs left? How about on day X-14, when you thought the major systems were all working but there was actually a major bug in the expected utility calculator? Or on day X-60, when a programmer got the Bayesian reasoning system working but it was connected to a stubbed-out version of the goal system instead of the real thing?
This myopic focus on boxing ideas misses most of the problems inherent in building a safe AGI.
I do. It implies that it is actually feasible to construct a text-only channel, which as a programmer I can tell you is not the case.
If you build your AI on an existing OS running on commercial hardware there are going to be countless communication mechanisms and security bugs present for it to take advantage of, and the attack surface of the OS is far too large to secure against even human hackers. The fact that you’ll need multiple machines to run it with current hardware amplifies this problem geometrically, and makes the idea that a real project could achieve complete isolation hopelessly naive. In reality you’ll discover that there was an undocumented Bluetooth chip on one of the motherboards, or the wireless mouse adapter uses a duel-purpose chip that supports WiFi, or one of the power supplies supports HomePNA and there was another device on the grid, or something else along those lines.
The alternative is building your own (very feature-limited) hardware, to run your own (AI-support-only) OS. In theory you might be able to make such a system secure, but in reality no one is ever going to give you the hundreds of millions of $$ it would cost to build the thing. Not to mention that a project that tries this approach will have to spend years duplicating hardware and software work that has already been done a hundred times before, putting it far behind any less cautious competitors...
Your second proposal, trying to restrict what the AI can do after it’s made a decision, is a lost cause. Our ability to specify what is and is not allowed is simply too limited to resist any determined effort to find loopholes. This problem afflicts every field from contract law to computer security, so it seems unlikely that we’re going to find a solution anytime soon.
Your first proposal, making an AI that isn’t a complete AGI, is more interesting. Whether or not it’s feasible depends partly on your model of how an AI will work in the first place, and partly on how extreme the AI’s performance is expected to be.
For instance, I could easily envision a specialized software engineering AI that does nothing but turn English-language program descriptions into working software. Such a system could easily devote vast computing resources to heuristic searches of design space, and you could use it to design improved versions of itself as easily as anything else. It should be obvious that there’s little risk of unexpected behavior with such a system, because it doesn’t contain any parts that would motivate it to do anything but blindly run design searches on demand.
However, this assumes that such an AI can actually produce useful results without knowing about human psychology and senses, the business domains its apps are supposed to address, the world they’re going to interact with, etc. Many people argue that good design requires a great deal of knowledge in these seemingly unrelated fields, and some go so far as too say you need full-blown humanlike intelligence. The more of these secondary functions you add to the AI the more complex it becomes, and the greater the risk that some unexpected interaction will cause it to start doing things you didn’t intend for it to do.
So ultimately the specialization angle seems worthy of investigation, but may or may not work depending on which theory of AI turns out to be correct. Also, even a working version is only a temporary stopgap. The more computing power the AI has the more damage it can do in a short time if it goes haywire, and the easier it becomes for it to inadvertently create an unFriendly AGI as a side effect of some other activity.
Actually, this would be a strong argument against CEV. If individual humans commonly have incoherent values (which they do), there is no concrete reason to expect an automated extrapolation process to magically make them coherent. I’ve noticed that CEV proponents have a tendency to argue that the “thought longer, understood more” part of the process will somehow fix all objections of this sort, but given the complete lack of detail about how this process is supposed to work you might as well claim that the morality fairy is going to descend from the heavens and fix everything with a wave of her magic wand.
If you honestly think you can make an AI running CEV produce a coherent result that most people will approve of, it’s up to you to lay out concrete details of the algorithm that will make this happen. If you can’t do that, you’ve just conceded that you don’t actually have an answer for this problem. The burden of proof here is on the party proposing to gamble humanity’s future on a single act of software engineering, and the standard of evidence must be at least as high as that of any other safety-critical engineering.
<A joke so hysterically funny that you’ll be too busy laughing to type for several minutes>
See, hacking human brains really is trivial. Now I can output a few hundred lines of insidiously convincing text while you’re distracted.
Yes, I’m saying that to get human-like learning the AI has to have the ability to write code that it will later use to perform cognitive tasks. You can’t get human-level intelligence out of a hand-coded program operating on a passive database of information using only fixed, hand-written algorithms.
So that presents you with the problem of figuring out which AI-written code fragments are safe, not just in isolation, but in all their interactions with every other code fragment the AI will ever write. This is the same kind of problem as creating a secure browser or Java sandbox, only worse. Given that no one has ever come close to solving it for the easy case of resisting human hackers without constant patches, it seems very unrealistic to think that any ad-hoc approach is going to work.
What I was referring to is the difference between:
A) An AI that accepts an instruction from the user, thinks about how to carry out the instruction, comes up with a plan, checks that the user agrees that this is a good plan, carries it out, then goes back to an idle loop.
B) An AI that has a fully realized goal system that has some variant of ‘do what I’m told’ implemented as a top-level goal, and spends its time sitting around waiting for someone to give it a command so it can get a reward signal.
Either AI will kill you (or worse) in some unexpected way if it’s a full-blown superintelligence. But option B has all sorts of failure modes that don’t exist in option A, because of that extra complexity (and flexibility) in the goal system. I wouldn’t trust a type B system with the IQ of a monkey, because it’s too likely to find some hilariously undesirable way of getting its goal fulfilled. But a type A system could probably be a bit smarter than its user without causing any disasters, as long as it doesn’t unexpectedly go FOOOM.
Of course, there’s a sense in which you could say that the type A system doesn’t have human-level intelligence no matter how impressive its problem-solving abilities are. But if all you’re looking for is an automated problem-solving tool that’s not really an issue.
I thought that too until I spent a few hours thinking about how to actually implement CEV, after which I realized that any AI capable of using that monster of an algorithm is already a superintelligence (and probably turned the Earth into computronium while it was trying to get enough CPU power to bootstrap its goal system).
Anyone who wants to try a “build moderately smart AGI to help design the really dangerous AGI” approach is probably better off just making a genie machine (i.e. an AI that just does whatever its told, and doesn’t have explicit goals independent of that). At least that way the failure modes are somewhat predictable, and you can probably get to a decent multiple of human intelligence before accidentally killing everyone.
The last item on your list is an intractable sticking point. Any AGI smart enough to be worth worrying about is going to have to have the ability to make arbitrary changes to an internal “knowledge+skills” representation that is itself a Turing-complete programming language. As the AGI grows it will tend to create an increasingly complex ecology of AI-fragments in this way, and predicting the behavior of the whole system quickly becomes impossible.
So “don’t let the AI modify its own goal system” ends up turning into just anther way of saying “put the AI in a box”. Unless you have some provable method of ensuring that no meta-meta-meta-meta-program hidden deep in the AGI’s evolving skill set ever starts acting like a nested mind with different goals than its host, all you’ve done is postpone the problem a little bit.
Why would you expect the social dominance of a belief to correlate with truth? Except in the most trivial cases, society has no particular mechanism that selects for true beliefs in preference to false ones.
The Darwinian competition of memes selects strongly for those that provide psychological benefits, or are politically useful, or serve the self-interest of large segments of the population. But truth is only relevant if the opponents of a belief can easily and unambiguously disprove it, which is only possible in rare cases.
If true, this is fairly strong evidence that the effort to turn the study of economics into a science has failed. If the beliefs of professional economists about their field of study are substantially affected by their gender, they obviously aren’t arriving at those beliefs by a reliable objective process.
Censorship is generally not a wise response to a single instance of any problem. Every increment of censorship you impose will wipe out an unexpectedly broad swath of discussion, make it easier to add more censorship later, and make it harder to resist accusations that you implicitly support any post you don’t censor.
If you feel you have to Do Something, a more narrowly-tailored rule that still gets the job done would be something like: “Posts that directly advocate violating the laws of in a manner likely to create criminal liability will be deleted.”
Because, you know, it’s just about impossible to talk about specific wars, terrorism, criminal law or even many forms of political activism without advocating real violence against identifiable groups of people.
I was commenting specifically about the end of your previous comment, not the whole topic. Sorry if that wasn’t clear. But as to this new point, why should an author feel obligated to gender-balance the complexity of the flaws they assign to minor characters?
Yes, I’m aware that there’s a fairly common intellectual position claiming that authors should devote vast amounts of energy to worrying about that sort of thing. I just think that’s a deeply misguided enterprise. A good author will naturally come to a pretty reasonable balance in the natural course of writing a story, and any major tweaking beyond that point is more likely to make the story worse than better.
Do you really think HP:MoR would be a better story if EY had spent a few weeks listing all the characters by gender, and trying to tweak the plot and insert details to ‘balance’ things? As opposed to, say, working out plot complications or dreaming up new moments of awesome?
So what?
From a storytelling perspective, authors are not obligated to make their main characters (or even 50% of main characters) female. Considering the way the whole SF&F genre has been taken over by gritty female urban fantasy vampire hunters in recent years, finding a decent story with a male lead is actually a nice change.
From the perspective of realism, the fact that the most competent characters are male is to be expected. That really is the way the world works, thanks to the fact that males have a flatter bell curve with longer tails on just about every measure of ability. It isn’t the result of an evil male conspiracy, and there’s nothing wrong with an author depicting this elementary fact of (current) human nature accurately.
So I’m left wondering how your comments amount to anything more than “I’m unhappy because you aren’t writing the story the way I would have done it.”
Knowing that philosophers are the only people who two-box on Newcomb’s problem, and they constitute a vanishingly small fraction of Earth’s population, I confidently one-box. Then I rush out to spend my winnings as quickly as possible, before the inevitable inflation hits.
Telling me what X is will have no effect on my action, because I already have that information. Making copies of me has no effect on my strategy, for the same reason.
Good insight.
No, even a brief examination of history makes it clear that the lethality of warfare is almost completely determined by the culture and ideology of the people involved. In some wars the victors try to avoid civilian casualties, while in others they kill all the adult males or even wipe out entire populations. Those fatalities dwarf anything produced in the actual fighting, and they can and have been inflicted with bronze age technology. So anyone interested making war less lethal would be well advised to focus on spreading tolerant ideologies rather than worrying about weapon technology.
As for the casualty rate of soldiers, that tends to jump up whenever a new type of weapon is introduced and then fall again as tactics change to deal with it. In the long run the dominant factor is again a matter of ideology—an army that tries to minimize casualties can generally do so, while one that sees soldiers as expendable will get them killed in huge numbers regardless of technology.
(BTW, WWI gases are nothing unusual in the crippling injury department—cannons, guns, explosives and edged weapons all have a tendency to litter the battlefield with crippled victims as well. What changed in the 20th century was that better medical meant a larger fraction of crippled soldiers to survive their injuries to return to civilian life.)