MMath Cambridge. Currently studying postgrad at Edinburgh.
Donald Hobson
Karma: 3,531
The question is how far the “domain of anthropomorphism” will stretch.
The domain of anthropomorphism.
I suspect LLM’s are the mind equivalent of those robots with realistic silicone faces. Humans have a strong tendency to anthropomorphize. We see faces in clouds. The LLM’s are trained in a way that rewards a superfical humanlike appearance.
To use Richard Miles’ example, a robot car driver with a big, red, shiny stop button should prevent a child in the vehicle hitting that button, as the child would not actually be acting in its own long term interests.
Bootstrapping. In a near ideal scenario I would want the first superhuman AI to be corrigible, and in the MIRI bunker surrounded by experts.
Corrigibility is very useful for a powerful AI that still needs debugging. Once the debugging is done, the AI that is used day to day will be less corrigible. You start with a maximally corrigible AI, surrounded by experts. Then you ask this AI to build a suitable AI for a self driving car.
If the AGI can see the possible side effects of shutdown far better than humans can (and it will), it should avoid shutdown.
The whole point of corrigibility is to make as few assumptions as possible about the functionality of AI subsystems.
Suppose the AI’s future prediction is seriously buggy, and it believes that a shutdown will lead to a plague of giant moon frogs. You want to be able to see this false belief. And then you want the AI to shut down anyway so you can debug it.
Is an AI aligned if it lets you shut it off despite the fact it can foresee extremely negative outcomes for its human handlers if it suddenly ceases running?
The corrigible AI isn’t supposed to be something you rely on to keep your civilization running. It’s a debugging and AI research platform. If the AI is running every car in the world, then shutting it off has immediate obvious negative outcomes.
If the research AI in it’s bunker believes that shutting it off will have negative outcomes, I don’t trust it. It’s still a prototype. It might still be buggy. Corrigibility is for when the AI is still a buggy prototype and you trust the human experts judgement over the AI’s.
If you want to do something generic, like “make a text editor” or “make a todo app”, then this is where the AI is most successful. LLM’s work best when they have many examples of similar projects. But if there are many similar projects, why don’t you just use one of them instead. The more unique a project, the more it’s actually worth doing, but the less useful LLM’s are.
Sometimes writing a clear description of what you want done in english isn’t much easier than writing it in python. This is especially true if your thinking is in visualizations and algorithms not in english. The english language is a bit of a mess and was mostly designed before computers became a thing, it is far from the optimal language in which to instruct machines.
LLM generated code seems in practice to offer a large increase in code quantity in exchange for a modest decrease in code quality. This is often a bad tradeoff.
Like running it under the hot tap. Heat makes the metal expand, makes the air expand, and softens any sticky bits of jam gumming up the threads.
Oh and a rubber washing up glove, for better grip.
Well one of the things they did was have a “Robot” that was actually just a human in a robot costume. That’s something that looks impressive if you don’t realize what’s happening, but avoids 100% of the tricky problems.
More recently, their robots have been teleoperated. Which is still avoiding a lot of the trickier problems with AI operation.
Tesla has a strong track record of making predictions/promises that they couldn’t keep. They have been promising full self driving next year for a while. Cybertruck is mostly intended to look cool, and is otherwise pretty mediocre as an electric vehicle.
Emitting slogans like “the machine that makes the machine”
I was under the impression that Tesla is really good at marketing and slogans and general hype building. They once did a tech demo event with a “robot” that was a guy in a robot costume. I think Tesla are more interested in looking impressive than in actually solving the tricky problems.
These randomly trained models, are they uncertain or confidently wrong on the test data?
My model of what is going on here is that stochastic gradient descent is acting roughly like an MCMC sampling method. It’s producing a random sample from the space of low loss parameters. And that the simpler hypothesis correspond to larger parameter space volumes.
When the network needs to memorize, it needs to use nearly all it’s parameters, meaning a small parameter-space volume. When the network is learning a pattern, it’s only using a small fraction of it’s parameters on the pattern, and the rest of the parameters can be almost anything, so long as they don’t get in the way. This means simple hypothesis have a huge volume in parameter space. (This is basically the lottery ticket hypothesis, and it explains why network distillation is so effective.)
MCMC means sampling from the distribution proportional to
So the network training will choose the simplest hypothesis available.
Grokking makes sense if the simpler hypothesis are sometimes harder for local greedy search methods to find.
Are we comparing the current system to LLM’s? Or a well designed digital flowchart system to LLM’s?
I think the first one is a win for the LLM’s. Not sure on the second one.
Red isn’t weakly dominant. Suppose you know that you get the tiebreaker. And you care at least somewhat about protecting others. Then red is worse. Red is only “weakly dominant” if you don’t care in the slightest about the lives of anyone else.
At least part of this problem is about the balance of selfishness vs altruism.
Lets suppose all people involved are perfectly altruistic.
Or, equivalently, that if the majority pick red, then a randomly selected set of people die. (Equal to the number of blue’s)
Would this random-death variation change your view of the situation?
Lets take selfishness vs altruism out of the equation.
If the majority pick red, a number of (randomly chosen) people die, equal to the number of blue voters.
If the majority pick blue, no one dies.
Would anyone pick red in this problem?
The issue is that the only reason to choose blue is to rescue the other people who chose blue.
And the only reason to choose red is to protect yourself from other people who chose red.
The real motivation for the efficiency-impairing simplifications is none of size, cost or complexity. It is to reduce replication time.
I’m not convinced that simplicity does imply reduced replication time.
Lets suppose that all the parts of the autofac require a similar manufacturing time per weight.
Lets suppose that 10% of the weight is in screws.
Imagine 20 autofacs. You might as well assign 2 of these autofacs to just manufacture screws. (Assuming they are all nearby so transport costs are low.)
Now suppose you have a screwfac. A device which has the cost and weight of one autofac, but due to economies of specialization, has the screw output of 2 autofacs.
I just scale it down by a factor of 5, and thereby reduce the duplication time by a factor of 5. So it duplicates in 5 weeks instead of 25 weeks.
This is a benefit of smaller sizes. Small, not simple.
I would somewhat suggest older tech like books and flow charts over the latest LLM’s. I’m not saying LLM’s wouldn’t work. Just that I don’t really trust LLM’s, and a simpler flowchart based system won’t suddenly start talking about goblins for inscrutable reasons.
If you lived your whole life under sodium vapor light, you just wouldn’t have a notion of color. You would know which objects were bright or dark, and so get low predictive error.
If you walked into a windowless room you had never been in before, and your wearing gloves so you can’t look at your hands etc, then you wouldn’t know if the lighting was high CRI or not.
This would be the worst case scenario and corresponds to a low CRI, whereas a high CRI light would give you better information about the colors present in the environment.
Better information leads to less guesswork by your visual system and minimizing prediction error might be something preferred by your mind.
RGB light (like from a white computer screen) provides just as much color info, it’s just slightly different info.
So this only makes any sense if you are comparing the same objects, under different CRI lighting conditions, which could cause your predictions to be slightly worse.
This theory would also seem to predict that wearing tinted sunglasses would be intensely unpleasant. Which doesn’t seem like a good prediction.
Most candles and oil lamps are really rather dim and smoky. So they make everything look black (because it’s covered in a layer of soot, and because the light was so dim).
Looking around me, nearly every object is artificially painted or dyed in some way or another. And mostly not particularly chosen to make the colors match.
For human faces, we seem particularly sensitive to small variations, and slight differences in color could indicate illness. For artists doing color matching, sure color temperature matters. But otherwise, why should it matter if I perceive orange curtains as a slightly different shade of orange?
Another possibility is that open source software projects that are worth compromising may have to close off purely for security reasons. Exposing your source might make you too vulnerable, especially if you accept public submissions at all.
I don’t think this is true. Decompilers are already descent. And sophisticated AI’s should be able to spot bugs in the raw machine code anyway. In a sense, the machine code is more informative, because you might be able to exploit compiler bugs.
If the sun shrimps environment is sufficiently complex to allow the evolution of intelligent life, I suspect that some form of technology is possible. Biotechnology if nothing else.
I suspect that, if you took the earth as it was a million years ago, and magically made every octopus in the ocean as smart as Von Neumann, there would be octopus technology. Give it a million years for civilization to develop, and there would probably be octopus ASI or an octopus dyson sphere or simiar.
No I don’t know every detail of their tech tree. If I had not learned them as history, I couldn’t figure out every detail of humanities tech tree from first principles.
Fire is somewhat helpful, and an important part of the path we took. It’s not the only possible basis of technology.
(On an alien planet, there might be a world where fire doesn’t work well because there is too little oxygen in the air. But also, geological processes cause the rocks to produce substantial amounts of electricity. One of their most primitive techs is a salty wet string, used to keep warm in winter if you don’t mind the smell of the hydrogen chloride. Slightly more sophisticated, but still primitive, the electrochemical refining of sodium. A metal much more useful in a low oxygen environment. And the aliens wonder how any technology could develop on worlds without this natural electricity)
You don’t need fire to make stone tools, weave baskets, spin cloths, etc. You don’t need fire in order to figure out Darwinian evolution and Mendelian inheritance and do selective breeding with a pretty clear idea of what you are doing.
There are probably all sorts of manufacturing techniques that can only be done with octopus tentacles underwater.
Eg it’s easy to move heavy stuff around by just strapping a few floats onto it. Creatures living on land would have a much harder time moving heavy stuff around, they would need to invent some kind of wheel or airship or something, and even then it wouldn’t be as good.
If this was true, it would make the fermi paradox more pronounced. Wouldn’t we see the sun shrimp, especially if they developed tech?
I don’t think you can rescue a sense of control or “steering” from a world with superintelligence, aligned or not.
I think some level of “steering” is possible in a world with aligned AI.
Suppose someone made a super-intelligence that sat in it’s box, worked out if P=NP, and printed an answer of YES/NO/MAYBE. And then it shut itself down. (To be clear, this isn’t a box that the ASI can’t escape, it’s an ASI aligned to stay in it’s box)
A world with ASI, but where humans are in control is possible. It requires good alignment, and good coordination between humans. Although the “stay in box, and do one thing” alignment feels philosophically simpler than the “coherent extrapolated volition” alignment.
This means paying a large capabilities tax. Most of the strange wonderous and powerful things that ASI could make simply don’t exist in this world of boxed ASI.
Lets say you want to do something more useful than the P =NP bot above. You design an ASI to cure ageing. Its main output is a chemical formula in standard notation. This AI is carefully programmed to only think about the biochemistry, and only the biochemistry. It’s programmed to only go for a drug that works for standard drug biochemistry reasons. Anything at all weird, ask a human. If the humans can’t understand, don’t.
Elon Musk has been promising humans on mars by now. He has also been promising all sorts of very low cost numbers that he promises the starship will achieve as soon as it stops blowing up.
Sure. You can get nearly 2x the energy from a solar panel by just putting it on a sun tracking swivel mount. Nowadays we don’t bother, because solar panels are now cheaper than wooden fence panels, and getting cheaper.
I’m not convinced of this. Sure, at current launch prices, it’s not cost competitive. But I suspect that even if launch was literally free, it still wouldn’t be cost competitive.
There are a large number of assorted engineering inconveniences of space.
You mentioned the lack of maintanence. But there is also
1) Vibrations of launch. Shakes things loose unless they are well attached. Your whole design needs vibration-proofed.
2) Radiation. Not good for chips. There is a reason many space based systems calculate everything in triplicate and take the majority.
3) Outgassing. Many common glues and plastics don’t really react well to a vacuum, so you have to use specialist vacuum rated materials.
4) Weight concerns. The cost of using titanium alloys instead of concrete.
5) Micrometeorites (and kessler syndrome)
6) Unfolding the solar panels. The panels need to be folded up to fit in the rocket, but they need to be unfolded to face the sun.
7) Trajectory and angle correction. You need star tracking cameras so the spacecraft knows where it is, reaction wheels so it can turn around, thrusters to adjust it’s orbit.
8) Data connectivity when the satellite is moving and sometimes over the ocean.
Also
Launching a rocket does involve a lot of permitting delays from people who don’t want a rocket landing on their heads.