MMath Cambridge. Currently studying postgrad at Edinburgh.
Donald Hobson
Karma: 1,932
But that almost never happens currently, so I don’t see why adding Excalidraw would change that.
From my experience of making a few diagrams for my posts, most of the work is in thinking what you want to represent, and making it. Not in saving and uploading a file. So I am predicting the rise in diagrams to be fairly small. Maybe as much as 50% more.
Ok, if you are planning to copy and paste existing software, that could involve less dev time. Or it could be integration hell. And the learning curve on it is pretty shallow. Such a feature would encourage more quick scribbly diagrams.
Its a vector drawing package. I don’t think it’s the best one out there. There are quite a few pieces of software people might use. A majority of the users won’t use this feature because their favourite software is better in some way. I mean if you have unlimited developer time, go for it. But I think the delta usability/ delta dev time is low. The software needs to have all the basic features and have a good UI before it starts offering any advantage at all. And half your users will still use other software, because that’s what they know how to use, or it has some obscure feature they like.
Here is a couple of “hard” things you can easily do with hypercompute, without causing dangerous consequentialism.
Given a list of atom coordinates, run a quantum accuracy simulation of those atoms. (Where the atoms don’t happen to make a computer running a bad program).
Find the smallest arrangement of atoms that forms a valid Or gate by brute forcing over the above simulator.
Brute forcing over large arrangements of atoms could find a design containing a computer containing an AI. But brute forcing over arrangements of 100 atoms should be fine, and can do a lot of interesting chemistry. Note that a psychoactive that makes humans care less about AI risk won’t be preferentially selected. Its not simulating the simple molecule in the world, that would be dangerous. Its simulating a simple molecule in a vacuum. (Or a standard temp and pressure 80% N2 + 20% O2 atmosphere, or some other simple hardcoded test setup.)
I think that the first paragraph after the block quote is highly confused.
Your actions depend on your utility function, the actions you have available and the probabilities you assign to various outcomes, conditional on various actions. Lets look at a few examples. (Numbers contrived and made up.)
These examples are deliberately constructed to show that expected utility theory doesn’t blindly output “Work on AI risk” regardless of input. Other assumptions would favour working on AI risk.
You are totally selfish, and are old. The field of AI is moving slowly enough that it looks like not much will happen in your lifetime. You have a strong dislike of doing anything resembling AI safety work, and there isn’t much you could do. If you were utterly confidant AI wouldn’t come in your lifetime, you would have no reason to care. But, probabilities aren’t 0. So lets say you think there is a 1% chance of AI in your lifetime, and a 1 in a million chance that your efforts will make the difference between aligned and unaligned AI. U(Rest of life doing AI safety)=1. U(wiped out by killer AI)=0, U(Rest of life having fun)=2 and U(Living in FAI utopia)=10. Then the expected utility of having fun is 2*0.99+0.01*x*10 and the expected utility of AI safety work is 1*0.99+0.01*(x+0.000001)*10 where x is the chance of FAI. The latter expected utility is lower.
You are a perfect total utilitarian, and highly competent. You estimate that the difference between galactic utopia and extinction is so large that all other bits of utility are negligible in comparison. You estimate that if you work on Biotech safety, there is a 6% chance of AI doom, a 5% chance of bioweapon doom, and the remaining 89% chance of galactic utopia. You also estimate that if you work on AI safety there is a 5.9% chance of AI doom and a 20% chance of bioweapon doom, leaving only a 74.1% chance of galactic utopia. (You are really good at biosafety in particular) You choose to work on the biotech.
You are an average utilitarian, taking your utility function to be U=pleasure/(pleasure+suffering) over all minds you consider to be capable of such feelings. If a galactic utopia occurs, its size is huge enough to wash out everything that has happened on earth so far leaving a utility of basically 1. You thing there is a 0.1% chance of this happening. You think humans on average experience 2x as much pleasure as suffering, and farm animals on average experience 2x as much suffering as pleasure, and there are an equal number of each. Hence in the 99.9% case where AI wipes us out, the utility is exactly 0.5. However, you have a chance to reduce the number of farm animals to ever exist by 10%, leaving a utility of (2+0.9)/(2+0.9+ 1+1.8)=0.509. This increases your expected utility by 0.009. An opportunity to increase the chance of FAI galactic utopia from 0.1% to 1.1% is only worth 0.005, (a 1% chance of going from U=0.5 to U=1) Therefore reducing the number of farm animals to exist takes priority.
1. Space colonization
There is some (small?) chance that the destruction will be limited to the Earth.
This chance is basically negligible, unless you made earth a special case in the AI’s code. But then you could make one room a special case by changing a few lines of code.
2. Mind uploading
With mind uploading, we could transmit out minds into outer space, with the hope that some day the data will be received by someone out there. No AGI can stop it, as the data will be propagated at the speed of light.
Probably no aliens anywhere near. (Fermi paradox) Human minds = Lots of data = Hard to transmit far.
The AI can chase after the signals, so we get a few years running on alien computers before the AI destroys those, compared to a few years on our own computers.
Runs risk of evil aliens torturing humans. Chance FTL is possible.
3. METI
If we are really confident that the AGI will kill us all, why not call for help?
We can’t become extinct twice. So, if we are already doomed, we can as well do METI.
If an advanced interstellar alien civilization comes to kill us, the result will be the same: extinction.
But if it comes to rescue, it might help us with AI alignment.If advanced aliens care, they could know all about us without our radio signals. We can’t hide from them. They will ignore us for whatever reason they are currently ignoring us.
4. Serve the Machine God
(this point might be not entirely serious)
In deciding your fate, the AGI might consider:
- if you’re more useful than the raw materials you’re made ofWith nanotech, the AI can trivially shape those raw materials into a robot that doesn’t need food or sleep. A machine it can communicate to with fast radio, not slow sound waves. A machine that always does exactly what you want it to. A machine that is smarter, more reliable, stronger, more efficient, and more suited to the AI’s goal. You can’t compete with AI designed nanotech.
At the nanotech stage, the AI can turn any atoms into really good robots. Self replication ⇒ Exponential growth ⇒ Limiting factor quickly becomes atoms and energy. If the AI is just doing self replication and paperclip production, humans aren’t useful workers compared to nanotech robots. (Also, the AI will probably disassemble the earth. At this stage, it has to make O’Neil cylinders, nanotech food production etc to avoid wiping out humanity.)
I think that given good value learning, safety isn’t that difficult. I think even a fairly halfharted attempt at the sort of Naive safety measures discussed will probably lead to non catastrophic outcomes.
Tell it about mindcrime from the start. Give it lots of hard disks, and tell it to store anything that might possibly resemble a human mind. It only needs to work well enough with a bunch of Miri people guiding it and answering its questions. Post singularity, a superintelligence can see if there are any human minds in the simulations it created when young and dumb. If there are, welcome those minds to the utopia.
No. The arguments look like a relatively small amount of ambiguous evidence, but what evidence is there doesn’t look good.
“If I thought the chance of AGI doom was smaller than the chance of asteroid doom, I would be working on asteroid deflection” is a common sentiment. People aren’t claiming tiny probabilities. They are claiming that its a default failure mode. Something that will happen (Or at least more likely than not) unless specifically prevented.
Looking at it from this perspective there must be binary falsification because any chance greater than 0 makes the argument ‘valid’, i.e. worth considering.
This reasoning is absurd. You are letting utilities flow back and effect your epistemics.
I think the notion of “falsification” as you state it is confused. In baysian probability theory, 0 and 1 are not probabilities, and nothing is ever certain. You start with some prior about the chance of AI doom, you read Nick Bostrums book and find evidence that updates these probabilities upwards.
How you act on those beliefs is down to expected utility.
Disjunctive style arguments tend to be more reliable than conjunctive arguments, for the same argument quality.
Like we know earth is round because.
Pictures from space.
Shadow on moon during lunar eclipse.
Combination of surface geographic measurements.
A sphere is the stable state of Newtonian attraction + pressure.
Ships going over horizon ⇒ Positive curvature. Only shape in euclidean 3d with positive curvature everywhere are topologically spheres. (Rules out doughnuts, not rounded cubes)
Other celestial objects observed to be sphere.
That is a disjunctive argument. A couple of the lines of evidence are weaker than others. Does this mean the theory is “unfalsifiable”. No. It means we have multiple reliable lines of evidence all saying the same thing.
Yes I have seen those creationist “100 proofs god exists”. The problem with those is not the disjunctive argument style, its the low quality of each individual argument.
Almost a tauutology = carries very little useful information.
In this case most of the information is carried by the definition of “Neuromorphic”. A researcher proposes a new learning algorithm. You claim that if its not neuromorphic then it can’t be efficient. How do you tell if the algorithm is neuromorphic?
If hypothetically that was true, that would be a specific fact not established by anything shown here.
If you are specific in what you mean by “brainlike” it would be quite a surprising fact. It would imply that the human brain is a unique pinnacle of what is possible to achieve. The human brain is shaped in a way that is focussed on things related to ancestral humans surviving in the savannah. It would be an enormous coincidence if the abstract space of computation and the nature of fundamental physical law meant that the most efficient possible mind just so happened to think in a way that looked optimised to reproductive fitness in the evolutionary environment.
It is plausible that the human brain is one near optimum out of many. That it is fundamentally impossible to make anything with an efficiency of > 100%, but its easy to reach 90% efficiency. The human brain could be one design of many that was >90%.
It is even plausible that all designs of >90% efficiency must have some feature that human brains have. Maybe all efficient flying machines must use aerofoils, but the space of efficient designs still includes birds, planes and many other possibilities.
I will claim that the space of minds at least as efficient as human minds is big. At the very least it contains minds with totally different emotions than humans, and probably minds with nothing like emotions at all. Probably minds with all sorts of features we can’t easily conceive of.
I was using that as a hypothetical example to show that your definitions were bad. (In particular, the attempt to define arithmetic as not AI because computers were so much better at it.)
I also don’t think that you have significant evidence that we don’t live in this world, beyond the observation that if such an algorithm exists, it is sufficiently non-obvious that neither evolution or humans have found it so far.
A lot of the article is claiming the brain is thermodynamically efficient at turning energy into compute. The rest is comparing the brain to existing deep learning techniques.
I admit that I have little evidence that such an algorithm does exist, so its largely down to priors.
Other effects I was considering.
Is the bottle rotationally symmetric? Is there say a weight of congealed shampoo in it?
If there was a slight tilt in this whole setup, the bottle could be marginally off vertical. Empty, the centre of gravity is quite high above the bar, and the slight tilt puts it slightly inward. With some water in the centre of gravity is lower. Full to the brim, the centre of gravity isn’t much lower.
There is also the slightly odd perspective that starts by saying 2 computers running the same computation only morally count once, and then goes on to claim that 2 battery hens are so mentally similar as to count as the same mind morally.
Some of these rhymes are just hard to decipher and would be better in clear English than bad poetry. I understand the desire to make a pithy saying, but it really isn’t clear what you meant with some of these.
Don’t dismiss these tasks just by saying they aren’t part of AGI by definition.
The human brain is reasonably good at some tasks and utterly hopeless at others. The tasks early crude computers got turned to were mostly the places where the early crude computers could compete with brains, ie the tasks brains were hopeless at. So the first computers did arithmetic because brains are really really bad at arithmetic so even vacuum tubes were an improvement.
The modern field of AI is what is left when all the tasks that it is easy to do perfectly are removed.
Suppose someone finds a really good algorithm for quickly finding physics equations from experimental data tomorrow. No the algorithm doesn’t contain anything resembling a neural network. Would you dismiss that as “Just traditional computer science”? Do you think this can’t happen?
Imagine a hypothetical world in which there was an algorithm that could do everything that the human brain does better, and with a millionth of the compute. If someone invented this algorithm last week and
AGI is defined as doing what the brain does very efficiently, not doing what computers are already good at.
Wouldn’t that mean no such thing as AGI was possible. There was literally nothing the brain did efficiently, it was all stuff computers were already good at. You just didn’t know the right algorithm to do it.
I think your thermodynamics is dubious. Firstly, it is thermodynamically possible to run error free computations very close to the thermodynamic limits. This just requires the energy used to represent a bit to be significantly larger than the energy dissipated as waste heat when a bit is deleted.
Considering a cooling fluid of water flowing at 100m/s through fractally structured pipes of cross section 0.01m^2 and being heated from 0C to 100C, the cooling power is 400 megawatts.
I think that superconducting chips are in labs today. The confidant assertion that superconductive reversible computing (or quantum computing) won’t appear before AGI is dubious at best.
Finally, have you heard of super-resolution microscopy https://en.wikipedia.org/wiki/Super-resolution_microscopy ? There was what appeared to be a fundamental limit on microscopes that was based on the wavelength of light. Physicists found several different ways to get images beyond that. I think there are quite a lot of cases where X is technically allowed under the letter of the mathematical equations, but feels really like cheating. This is the sort of analysis that would have ruled out the possibility. (And did rule out a similar possibility of components far smaller than a photons wavelength communicating with photons) This kind of rough analysis can easily prove possibility, but it takes a pedant with the security mindset and a keen knowledge of exactly what the limits say to show anything is impossible. So not only are there reversible computing and quantum computing, there are other ideas out there that skirt around physical limits on a technicality that haven’t been invented yet.
Suppose someone in 1900 looked at balloons and birds and decided future flying machines would have wings. They called such winged machines “birdomorphic”, and say future flying machines will be more like birds.
I feel you are using “neuromorphic” the same way. Suppose it is true that future computers will be of a Processor In Memory design. Thinking of them as “like a brain” is like thinking a fighter jet is like a sparrow because they both have wings.
Suppose a new processor architecture is developed, its basically PIM. Tensorflow runs on it. The AI software people barely notice the change.
Just pointing out that humans doing arithmetic and GPT3 doing arithmetic are both awful in efficiency compared to raw processor instructions. I think what FeepingCreature is considering is how many other tasks are like that?
You may have a point. The trivial inconvenience effects are probably large for some people.