MMath Cambridge. Currently studying postgrad at Edinburgh.
Donald Hobson
Karma: 3,507
These randomly trained models, are they uncertain or confidently wrong on the test data?
My model of what is going on here is that stochastic gradient descent is acting roughly like an MCMC sampling method. It’s producing a random sample from the space of low loss parameters. And that the simpler hypothesis correspond to larger parameter space volumes.
When the network needs to memorize, it needs to use nearly all it’s parameters, meaning a small parameter-space volume. When the network is learning a pattern, it’s only using a small fraction of it’s parameters on the pattern, and the rest of the parameters can be almost anything, so long as they don’t get in the way. This means simple hypothesis have a huge volume in parameter space. (This is basically the lottery ticket hypothesis, and it explains why network distillation is so effective.)
MCMC means sampling from the distribution proportional to
So the network training will choose the simplest hypothesis available.
Grokking makes sense if the simpler hypothesis are sometimes harder for local greedy search methods to find.
Are we comparing the current system to LLM’s? Or a well designed digital flowchart system to LLM’s?
I think the first one is a win for the LLM’s. Not sure on the second one.
Red isn’t weakly dominant. Suppose you know that you get the tiebreaker. And you care at least somewhat about protecting others. Then red is worse. Red is only “weakly dominant” if you don’t care in the slightest about the lives of anyone else.
At least part of this problem is about the balance of selfishness vs altruism.
Lets suppose all people involved are perfectly altruistic.
Or, equivalently, that if the majority pick red, then a randomly selected set of people die. (Equal to the number of blue’s)
Would this random-death variation change your view of the situation?
Lets take selfishness vs altruism out of the equation.
If the majority pick red, a number of (randomly chosen) people die, equal to the number of blue voters.
If the majority pick blue, no one dies.
Would anyone pick red in this problem?
The issue is that the only reason to choose blue is to rescue the other people who chose blue.
And the only reason to choose red is to protect yourself from other people who chose red.
The real motivation for the efficiency-impairing simplifications is none of size, cost or complexity. It is to reduce replication time.
I’m not convinced that simplicity does imply reduced replication time.
Lets suppose that all the parts of the autofac require a similar manufacturing time per weight.
Lets suppose that 10% of the weight is in screws.
Imagine 20 autofacs. You might as well assign 2 of these autofacs to just manufacture screws. (Assuming they are all nearby so transport costs are low.)
Now suppose you have a screwfac. A device which has the cost and weight of one autofac, but due to economies of specialization, has the screw output of 2 autofacs.
I just scale it down by a factor of 5, and thereby reduce the duplication time by a factor of 5. So it duplicates in 5 weeks instead of 25 weeks.
This is a benefit of smaller sizes. Small, not simple.
I would somewhat suggest older tech like books and flow charts over the latest LLM’s. I’m not saying LLM’s wouldn’t work. Just that I don’t really trust LLM’s, and a simpler flowchart based system won’t suddenly start talking about goblins for inscrutable reasons.
If you lived your whole life under sodium vapor light, you just wouldn’t have a notion of color. You would know which objects were bright or dark, and so get low predictive error.
If you walked into a windowless room you had never been in before, and your wearing gloves so you can’t look at your hands etc, then you wouldn’t know if the lighting was high CRI or not.
This would be the worst case scenario and corresponds to a low CRI, whereas a high CRI light would give you better information about the colors present in the environment.
Better information leads to less guesswork by your visual system and minimizing prediction error might be something preferred by your mind.
RGB light (like from a white computer screen) provides just as much color info, it’s just slightly different info.
So this only makes any sense if you are comparing the same objects, under different CRI lighting conditions, which could cause your predictions to be slightly worse.
This theory would also seem to predict that wearing tinted sunglasses would be intensely unpleasant. Which doesn’t seem like a good prediction.
Most candles and oil lamps are really rather dim and smoky. So they make everything look black (because it’s covered in a layer of soot, and because the light was so dim).
Looking around me, nearly every object is artificially painted or dyed in some way or another. And mostly not particularly chosen to make the colors match.
For human faces, we seem particularly sensitive to small variations, and slight differences in color could indicate illness. For artists doing color matching, sure color temperature matters. But otherwise, why should it matter if I perceive orange curtains as a slightly different shade of orange?
Another possibility is that open source software projects that are worth compromising may have to close off purely for security reasons. Exposing your source might make you too vulnerable, especially if you accept public submissions at all.
I don’t think this is true. Decompilers are already descent. And sophisticated AI’s should be able to spot bugs in the raw machine code anyway. In a sense, the machine code is more informative, because you might be able to exploit compiler bugs.
If the sun shrimps environment is sufficiently complex to allow the evolution of intelligent life, I suspect that some form of technology is possible. Biotechnology if nothing else.
I suspect that, if you took the earth as it was a million years ago, and magically made every octopus in the ocean as smart as Von Neumann, there would be octopus technology. Give it a million years for civilization to develop, and there would probably be octopus ASI or an octopus dyson sphere or simiar.
No I don’t know every detail of their tech tree. If I had not learned them as history, I couldn’t figure out every detail of humanities tech tree from first principles.
Fire is somewhat helpful, and an important part of the path we took. It’s not the only possible basis of technology.
(On an alien planet, there might be a world where fire doesn’t work well because there is too little oxygen in the air. But also, geological processes cause the rocks to produce substantial amounts of electricity. One of their most primitive techs is a salty wet string, used to keep warm in winter if you don’t mind the smell of the hydrogen chloride. Slightly more sophisticated, but still primitive, the electrochemical refining of sodium. A metal much more useful in a low oxygen environment. And the aliens wonder how any technology could develop on worlds without this natural electricity)
You don’t need fire to make stone tools, weave baskets, spin cloths, etc. You don’t need fire in order to figure out Darwinian evolution and Mendelian inheritance and do selective breeding with a pretty clear idea of what you are doing.
There are probably all sorts of manufacturing techniques that can only be done with octopus tentacles underwater.
Eg it’s easy to move heavy stuff around by just strapping a few floats onto it. Creatures living on land would have a much harder time moving heavy stuff around, they would need to invent some kind of wheel or airship or something, and even then it wouldn’t be as good.
If this was true, it would make the fermi paradox more pronounced. Wouldn’t we see the sun shrimp, especially if they developed tech?
I don’t think you can rescue a sense of control or “steering” from a world with superintelligence, aligned or not.
I think some level of “steering” is possible in a world with aligned AI.
Suppose someone made a super-intelligence that sat in it’s box, worked out if P=NP, and printed an answer of YES/NO/MAYBE. And then it shut itself down. (To be clear, this isn’t a box that the ASI can’t escape, it’s an ASI aligned to stay in it’s box)
A world with ASI, but where humans are in control is possible. It requires good alignment, and good coordination between humans. Although the “stay in box, and do one thing” alignment feels philosophically simpler than the “coherent extrapolated volition” alignment.
This means paying a large capabilities tax. Most of the strange wonderous and powerful things that ASI could make simply don’t exist in this world of boxed ASI.
Lets say you want to do something more useful than the P =NP bot above. You design an ASI to cure ageing. Its main output is a chemical formula in standard notation. This AI is carefully programmed to only think about the biochemistry, and only the biochemistry. It’s programmed to only go for a drug that works for standard drug biochemistry reasons. Anything at all weird, ask a human. If the humans can’t understand, don’t.
In practice decisions of CEOs of large corporations routinely lead to harming a great lot of people and they get very minor reprecussions for it if any.
True. But there is also a kind of scaling error here.
Suppose you run a small business with one employee. You ask your employee to do something slightly risky. Most of the time it works out fine. And if it doesn’t, it’s a tragic freak accident.
Now scale up. Your business now employs millions of people. Someone is dying from the job every few days.
Any tradeoff on sufficiently large scales is going to have many lives on both sides of the equations. Which makes it very easy to paint the CEO’s as evil mass murderers if you ignore the other side of the tradeoff.
And there’s a minimum amount of money that makes you a danger to society, able to buy laws, screw over communities and so on;
Fair, but. There is a minimum amount of money that lets you start your own space program just because you think rockets are cool. And that amount is several billion.
This would also mean mandatory dilution of corporate control: when a company gets big enough to push on society, the public should get a bigger and bigger say in how it’s run, with founders and investors keeping enough control to be ultra rich but not enough to run their bulldozer over society.
It sounds like under this system, Amazon would be a moderately successful online book store that was run by a squabbling mess of politicians.
Or perhaps Bezos would carefully keep his company below the limit so as to stay in control.
Power would be divided towards more smaller companies. Which would mean different perverse incentives, but maybe not fewer perverse incentives? I’m not convinced that lots of little companies is better than one big company.
as without allowing break of journeys you would only be able to do the journey A-D with an A-D ticket, as the railways intended.
You get on the train from A to D for a job interview. However halfway through your journey you get a text telling you that someone else has the job, don’t bother showing up. So you get off at the next station, C.
People can and do have changes of plans. This is going to end with a railway worker trying to stop a passenger from exiting. Because the A to C journey is more expensive than the A to D journey, and the passenger only paid for A to D. And so the railway staff try to force the passenger onto the train from C to D. And the passenger accuses the train company of kidnapping them. That is going to be a mess.
That or the rule is written, but unenforced and routinely flouted.
If the railway company doesn’t like this, they can assign prices to make this impossible. A simple per mile fee would work. Or at least, assign a positive number to each little segment of journey, (two consecutive stations). Assign the price for longer journeys to be the sum of these pieces. Assign the value of flexible tickets to be the maximum value of all the individual journeys they could be used on.
I disagree that wack-a-mole isn’t winnable.
Every dollar of tax revenue that the treasury revieves is, in a sense, a win in a game of wack-a-mole with the tax dodgers.
From inside the system, you can’t put your feet up, you need to keep wacking away. But the same could be said for picking weeds in a garden. Still, with skill and hard work, you can keep the problem mostly under control.
The Player still has moves; in general they’ll try to get their land assessed for only a small fraction of its value and lower their taxes that way, but the point still stands. And even that exploit can be curtailed if land valuations are made public, because it’ll be really obvious to everyone that the land owned by the billionaire who just had lunch with the President is worth way less than it should be.
Land Value Taxes, and Property Taxes in general, are a much better source of government revenue than income taxes, because they’re impossible to dodge the way that income taxes can be dodged.
I think you just aren’t being inventive enough to think of all the loop holes here.
One trick is Banach Tarski tax dodging. You divide your land into a large number of small regions of incredibly intricate shape, to which no value can be easily assigned.
For example, divide your house into 100 pieces, each owned by a different shell company. Each piece is low value (because who wants a sliver of someone elses bathroom) The pieces of land are only useable when all owned by the same person.
If the government doesn’t buy this, and values the pieces so they add up to a normal house price. Then you could always try only paying tax on a few of the pieces. The rest get auctioned off, but no one wants random pieces of bathroom, so they sell cheap.
Then there is the difficulty of separating the value of the land from the value of the stuff on the land.
Land can be more valuable, or less valuable, for all sorts of subtle reasons. (Zoning rules, subsidence problems, rare bats, etc)
Consider an empty field near a town. If you could build houses on it, those houses would be valuable. But there are zoning rules that say any project needs “community approval”. So now the valuators have to decide if the community would approve of a block of flats or not.
An empty patch of land in central new york is more valuable than the same patch in the middle of nowhere. Because of the improvements (like shops) that other people have made to nearby patches of land.
So if one company owned the whole of new york, they could argue with a straight face that, absent their improvements (the whole city) it would just be another almost worthless bit of field. So this favors one company owning an entire city, and leasing it out to everyone else.
Land value can be reduced by having bad neighbours.
What about land that comes with rules/contracts. The owner of this land must not mess with the water main that runs under their house. They must allow pedestrian access to a footpath through their garden. Etc. Can a really bad Home Owners Association reduce the value of a piece of land?
There are a lot of complexities to play with here. It’s inevitably going to end up as a mess of loopholes. We just don’t know what specific mess yet.
If you slowly learned each one of these worlds in order[1], every new world would be a huge surprise that reframed everything before it.
I’m not sure this is the case. I feel like once you learned All maths, physics would be just “see these equations, they are real (What does real mean, who knows)”
And then consciousness would be just “you see those monkeys over on the third planet from the sun”
I think that what you mean is a basic maths textbook won’t go into detail on the equations of quantum field theory, because it’s too complicated. And a basic physics textbook won’t discuss psychology in terms of the fundamental particles that make up a brain, again, because it’s too complicated.
So, do you see maths as a vast infinite abstract structure, or just the stuff that’s simple enough to be in the textbook?
The more we drill down into what we mean by behaves exactly like water, the more it starts to become clear that there just isn’t a possible substance which behaves exactly like water, but isn’t. There are only so many configurations of electrons and protons and neutrons
When Imagining alternate laws of physics, we could imagine that the molecules had tiny XML tags on them that said “Totally XYZ not water”. And that these tags were basically epiphenomenal.
Or at least, we could imagine some extra conserved quantity/force particle thing. Some extra particle that could stick to a nucleus, but not really do much.
In reality, there is heavy water, which is pretty similar to regular water, but slightly heavier. (Especially similar if it’s the oxygen that has an extra neutron).
The actual price of the goods and services, in the sense of “what does it take to provide them?” has gone up, and the market price will necessarily follow.
In some, admitedly somewhat toy, scenarios, price gouging laws can help the consumer (at the expense of the seller)
Imagine that there are 200 people, each of whom are hungry enough that they are indifferent to paying $100 for a can of beans. The shop has 100 cans of beans in the basement. They fly in another 100 cans by helicopter at the cost of $90/can. They set the price to $99.99. Everyone gets fed, but they pay dearly for it. Consumer surplus, ~0.
Now suppose price gouging laws exist. 100 of those people turn up to empty shelves. But the other 100 get a can of beans for $2, and so get $98 surplus. So on average, consumer surplus has increased.
\cut
Lets change the scenario slightly. There are now 300 people. 200 of those people can barely afford to pay $100 for the beans. But 100 of those people are rich, and can afford $500 for the beans. Now there is no helicopter. There are 200 cans of beans in the store. The profit maximizing thing for the store to do is to set the price to $499, and to not sell half the cans (at least until the disaster is over. Or alternatively, the rich buy 2 cans each and have a bean feast). If you are a poor person, your chance of getting any beans is 0. Whereas with price gouging laws, it’s first come first served. So you have a 2⁄3 chance of getting beans.
Are these contrived toy examples. Maybe. But I think they are at least coherent. If the elasticity of supply with price is low, gouge pricing can favor whoever was lucky enough to have a big stockpile (usually the stores), and can sometimes favor the convenience of the rich over the life of the poor.
I was under the impression that Tesla is really good at marketing and slogans and general hype building. They once did a tech demo event with a “robot” that was a guy in a robot costume. I think Tesla are more interested in looking impressive than in actually solving the tricky problems.