avturchin
Karma: 592
NewTop
If an AI observers strong inconsistency in my liking-wanting-approving, should it stop (and inform me about it), or try to agregate my preference anyway?
Probably the difference is in the angle of slope of the availability of the drug. For a person who never injected anything in his system (like me), going to the “right part of the city” is frightful experience which I would hopely never do. It is very hard step. But not the same with overeating or internet-addiction, where entrance is very easy.
“once monitoring equipment detects a problem you can simply shut off the intervention”
Look on the Facebook addiction—did the Facebook ever shutdown anyone because he spent too long on their site and was procrastinated at work? Some people block FB on their computers voluntary, but they still lurk through proxies—it is not easy to block something harmful but addictive.
Anyway, I understand your desire to “good wireheading” which will end sufferings and will not hinder productivity, and some AI controlled brain stimulation may be such a system. But if it is not controlled by advanced AI, but by just a few parameters regulation, a person could easily unwillingly touch a region of the brain (and utility function) from which he will become instantly severely addicted.
Wireheading as a Possible Contributor to Civilizational Decline
My mom told me that if she wrote less than 10 page a day, she was unproductive, but if she wrote more than 10 pages, all above the tenth page was garbage, and had to be deleted next day. I don’t know if this random memory will be helpful or not.
Sometimes winning is an evidence of non-rationality. For example, if one plays in a lottery and wins a million dollars, - it was still irrational for him to play as most lotteries have negative total expected utility. The same thing is with becoming very rich: most who try, fail.
Imagine the following game: You are put into a bath where you will be a) dissolved with acid with 99 per cent probability and b) you will become a billionaire with 1 per cent. Would you agree to play?
I would say that playing the game is very irrational, and any winner was likely not able to correctly calculate the odds. So extreme winning is a signal of some form of irrationality.
One can’t understand code, but predicting the goals of the programmer may be a simpler task. If he has read “Superintelligence”, googled “self-improving AI” and is an expert in ML, the fact that he locked himself into a basement may be alarming.
Interestingly, humans are able to predict each other values in most cases—and this helps our society to exist. Relationship, market, just walking out—all it is based on our ability to read the intentions of other people successfully.
However, many cases of bad events happen when we don’t understand each other intentions: this enable scammers and interpersonal conflicts.
Reading this I had an idea about using the reward hacking capability to self-limit AI’s power. May be it was already discussed somewhere?
In this setup, AI’s reward function is protected by a task of some known complexity (e.g. cryptography or a need to create nanotechnology). If the AI increases its intelligence above a certain level, it will be able to hack its reward function and when the AI will stop.
This gives us a chance to create a fuse against uncontrollably self-improving AI: if it becomes too clever, it will self-terminate.
Also, AI may do useful work while trying to hack its own reward function, like creating nanotech or solving certain type of equations or math problems (especially in cryptography).
Thanks for participating in interesting conversation which helped me to clarify my position.
As I now see, the accelerated growth, above Moore’s law level, started only around 2016 and is related not to GPU, which grew rather slowly, but is related to specialised hardware for neural nets, like Tensor cores, Google TPU and neuromorphic chips like True North and Akida. Neuromorphic chips could give higher acceleration for NNs than Tensor cores, but not yet hit the market.
Huang law is marketing, but Moore’s law is also marketing. However, the fact that DGX-2 outperform the previous DGX-1 10 times in performance and 4 times in cost efficiency in deep learning applications after 1 year implies that some substance is real here.
The GPU performance graph you posted is from 2013 and is probably obsolete; anyway, I don’t claim that GPU is better than CPU in term of FLOPS. I point on the fact that GPU and later TPU (which not existed in 2013) are more capable on much simpler operations on large matrix, which, however, is what we need for AI.
AI impact article is also probably suffer from the same problem: they still count FLOPS, but we need TOPS to estimate actual performance in neural nets tasks.
Update: for example, NVIDIA in 2008 had top cards with performance around 400 gigaflops, and in 2018 - with 110 592 gigaflop in Tensor operations (wiki) which implies something like 1000 times growth in 10 years, not “order of magnitude about every 10-16 years”. (This may not add up to previous claim of 4-10 times growth, but this claim applies not to GPU, but to TPU—that is specialised ASICs to neural net calculations, which appeared only 2-3 years ago and the field grows very quickly, most visibly in form of Google’s TPU.)
FLOPS is a bad measure for neural net training, as NNs typically employ simpler operations with less precision. “TOPS”—trillion operation per second—is widely used now to measure them.
“4-10 times a year growth” is Huang law which was formulated by NVIDIA CEO Huang about the speed of growth of computation in GPU Link.
An example of this law is NVIDIA’s computer DGX-2 released in March 2018 for $400K with 2 Petaflop in deep learning performance which is said to be 10 times faster in neural nets training than the system DGX-1 from 2017 which cost $149K (this means 4 times increase of cost effectiveness in 1 year). Source.
The same way Google’s TPU is growing very quickly:
TPU 1 generation − 2016 − 92 TOPS, but inference only (that is no learning)
TPU 2 gen − 2017 − 180 teraflops in deep learning
TPU 3 gen − 2018 − 8 times performance than TPU2, link
I didn’t get why you think that NN specific hardware will produce only 40x improvement in the next years. There is several ways to dramatically increase NN hardware beyond tensor processing units using even more specialised systems:
“· Intel has promised to increase neural net performance 100× by 2020 (from 2017) by use of specialized chip-accelerators they called Nervana.
· 3D chips. A 3D System combining memory and computing cores on a chip may increase energy efficiency 1000 times and computational speed more than 50 times by 2021 by eliminating memory bottlenecks, according to DARPA.
· FPGA. These programmable chips could combine the efficiency of TPUs with the speed of ordinary computers. Fujitsu claims to have optimized FPGA architecture to be 10,000 times faster (Fujitsu, 2016).
· Memristors. Memristors seem to enable more efficient neural networks (Du et al., 2017; Kaplan, Yavits, & Ginosar, 2018). They could be the basis for physical neural nets, which could be especially effective in inference, as each memristor will replace one synapse.
· Spiking neural nets. The TrueNorth chip from IBM provides 10 000× the energy economy of conventional chips and could solve the same tasks as ordinary neural nets after compilation (Hsu, 2014). IBM also invented in 2018 a system of analogues synapses, which provides 100 times the power economy, and also impose less load on the information transfer bus, as the synapses are trained “locally”, as in the human brain (Ambrogio et al., 2018).
· Non-von-Neumann architectures. DARPA is exploring a new type of computing which could offer a 1000× boost in computational power called HIVE which will “its ability to simultaneously perform different processes on different areas of memory simultaneously” and work with data graphs (Johnson, 2017).” (self cite from a draft on the topic).
-In memory computing could provide 100 times growth link.
This all doesn’t take into account several other ways to increase performance, like quicker running of programs on existing hardware via special languages and possible progress in algorithms.
Moreover, if we ignore all this evidence, but take the claim about the neuromorphic chip with 1 million neurons like Akida for its face value—and it is likely that such chips will appear in early 2020s—than performance of such single chip, given its possible speed of hundreds megahertz, will be comparable with some estimations of the human brain performance:
Roughly: million neurons running million times quicker than human ones = performance of 1 trillion of neurons, that is more than human brain.
Obviously, it is not a proof of AGI capability, but the situation than a singe chip has performance comparable with human brain is a prompt to worry and it is reasonable to give 10 per cent probability to AGI appearing in next 10 years.
We should adjust not to general Moore’s law, but to the speed of performance increase in specialised neural net hardware, which is currently around 4-10 times a year, in form of different TPU, ASICs and neuromorphic chips, like Akida, which has 1 million neurones and 10 billion synapses.
Extrapolating such trends gives roughly human level performance in reach of AI-scientists somewhere in 2020s. It may not require new technological process in a sense of lithography, as the amount of (AI-related)-computation will increase inside a single chip via different architectural tricks, not via shrinking of transistors.
Also, given arms race in AI, I would not be surprise that the biggest players will be able to spend tens of billions of dollars on hardware.
Finally, “7 years” is within the predicted period of 10 years where AGI may appear, that is, even if the fast hardware increase will end, the resulting hardware overhang may be enough to run very powerful AIs.
I think you are missing that AI related performance and hardware growth has a discontinuity moment around 2012 and is growing now in many important metrics with speed 10 times a year (or doubling time of 3.5 months, see e.g. OpenAI’s “AI and compute”). I collected other evidence for 10 per cent in 10 years here.
Possible Dangers of the Unrestricted Value Learners
[retracted]
Maybe visibility of UDASSA is small? Even I, despite lurking on LW for 10 years, didn’t recognise it immediately. As I know, there is no proper scientific article about Wei Dai’s theory and most of it is presented on a half-dead site where some interesting links, like “history of UDASSA” are dead. Also, UDASSA can’t be find using Google scholar.
However, Muller doesn’t mentioned the dust theory either.
You are right, it is problem, but I suggested a possible patch in Update 2: The idea is to create copies not of S(t) moment, but only of the next moment S’(t+1), where the pain disappears and S is happy that he escaped form eternal hell.
In your example, it will be like to procreate healthy children, but tell them that their life was very bad in the past, but now they are cured and even their bad memories are almost cured (it may seem morally wrong to lie to children, but it could be framed as watching a scary movie or discussing past dreams. Moreover, it actually works: I often had dreams about bad things happened to me, and it was a relief to wake up; thus, if a bad thing will happen to me, I may hope that it is just a dream. Unfortunately, it not always works.)
In other words, we create indexical uncertainty not about my current position but about my next moment of experience.
Yes, it’s a pity that Wei Dai’s work was not mentioned. However, it looks like that Muller and Wei Dai tried to answer different questions. Wei Dai wrote about decision theories, and Muller tries to derive foundations of quantum mechanics and other physical laws from relation of observer-moments.
Yes, other types of “preferences” are conceivable. For example, if a person is acting under an order of another person, like a soldier, he may not like, nor want or approve the order, but still obey it, as he has to.