No, and that is a good idea.
Algon
Karma: 4,074
A bad review != a bad book
A pencil is not a pencil is not a pencil
Tangentially relevant: Armstrong has a post on some general features that make it likely that Goodharting will be a problem. Conversely, he gives some features that make it less likely you’ll wind up Goodharting in your search.
So, as long as:
We use a Bayesian mix of reward functions rather than a maximum likelihood reward function.
An ideal reward function is present in the space of possible reward functions, and is not penalised in probability.
The different reward functions are normalised.
If our ideal reward functions have diminishing returns, this fact is explicitly included in the learning process.
Then, we shouldn’t unduly fear Goodhart effects (of course, we still need to incorporate as much as possible about our preferences into the AI’s learning process). The second problem, making sure that there are no unusual penalties for ideal reward functions, seems the hardest to ensure.
Liouville’s Theorem and the Second Law
In the roguelike Angband, the player has to descend 100 floors of the titular dungeon to kill Morgoth and beat the game. Novices hang about the shallower floors, slaying mini-bosses, gaining levels, collecting loot and not descending to further levels. Inevitably, they die well before reaching floor 100. Old hands advise against this and say to beat the game you need to focus on going down. Inevitably, novices do not listen until they’ve died countless times and beaten the game. At which point, the cycle repeats.
All of that is to say: it’s true that you need to play to win. It’s also true that people don’t get this. One exception is people who’ve played to win in the past, usually after many failures as they stubbornly ignore advice on how to win.
I should know, as I was that novice in Angband at one point. Eventually, I gave in and gave in to victory, descending as fast as I could handle. For quite some time, this was difficult. Even though I explicitly aimed to win, for some reason I would eventually just lose the ability to focus on descending and start puttering about collecting loot like a magpie. Rest was required before I could get my head back in the game. If I didn’t, the result was death.
Why exactly my ability to play to win could be sapped so easily confused me. I wasn’t sure if it was because of bad habits, or because of some generalized failure to follow an explicit goal or what. So once I beat Angband, I decided to determine why I struggled to win by playing another roguelike where you win by descending through a dungeon: Noita.
And what do you know, I still struggled to play to win! I still struggled to descend or execute on any strategy like “acquire as much health as possible”, “beat this dude to unlock this spell for future runs” etc. So it wasn’t bad habits that killed me in Angband, but a generalized difficulty in optimizing for a goal for an extended period of time.
Perhaps then playing to win is like a resource you can spend, and most people just don’t care to spend that resource on board games. More optimistically, playing to win is like a muscle you can train. In which case, you want to play to win at board games.
I’m not sure which is true. But either way, everyone should experience of playing to win at least a few times in their lives, till they learn what it feels like and can tell when they’re playing to win, and when they’re not.
Continuous takeoff is a bad name
Some interesting books from my reading list.
Analytical Mechanics—Nivaldo A LemsIn the tradition of V.I. Arnold, Goldstein, and Landau & Lifshitz. The author is really classical mechniacs pilled. Though the book is more focused on the mathematics of it all than the physics.
Fundamental Principles of Classical Mechanics
In the same tradition as Lems but more heuristic. It uses physical motivation to build up to formal ideas. Places a greater focus on a local co-ordinate approach. The author emphasises Cartan’s Method of Moving Frames, for pedagogical purposes. This is linked to the historical development of the subject, which I quite like. But notably, the author says Goldstein is the gold standard for the mathematical methods of classical mechanics. So why bother reading this book?Differential Geometry in Physics—Gabriel Lugo
Has coloured figures. La-dee-da.
Pre-reqs: Calculus, Linear Algebra and Differential Equations. Assumes the reader has encountered the concepts in classical/quantum mechanics.
Aims to:
1) Bridge differential geometry’s historical practical formulation and its abstract modern form.
2) Help physicists who want to strengthen their mathematical ability. The author was shocked that HPE folk and differential geometry folk worked on connections on fiber bundles whilst unaware that they were really working on the same stuff. This was part of the inspiration of this book, meant to rectify the situation.
3) Be readable to those at the boundary of maths/physicists.
4) Serve as background for GR.
5) Add material that is unusual for a textbook, such as solitions on the style of Backlund transform instead of Olver.Geometry from dynamics: classical and quantum.
Looks interesting, certainly a different perspective.
Something about dynamics giving rise to geometric structure e.g. Hamiltonian, Lagrangian and Symplectic geometries.
Claims geometry is experienced first over dynamics.Relativity and Cosmology—Thorne & Blandford
You know you want it.
Emphasises a geometric approach.
Takes 1/4-1/2 of a semester to teach.
Emphasises historical transformations in the physics of GR.
Has practical calculations!Structure and Interpretation of Classical Mechanics − 2nd Edition.
1) Uses Scheme. Apparently they don’t need to teach it, as students just learn it in a matter of days.
2) Notation is built to make turning equations into programs easy. The author claims this simplifies things as a result.
3) Focuses on motion over deriving equations of motion.
3) Non-linear dynamics is given a lot of time throughout the text.
4) Lots of time spent on phase space perspective.
5) This book has many cool qualitative results.Concise Treatise on Quantum Mechanics in Phase Space
1) This book focuses on Wigner’s formulation of quantum mechanics. Wigner’s Quasi-Probability distribution function in phase space furnishes a representation of quantum mechanics on equal footing to the Hamiltonian and Path Integral formulation.
2) The author claims this representation more naturally and intuitively connects to the classical limit.
3) The representation is weird. You have simultaneous position and momentum variables
4) The theory was built by two unknown researchers, and faced opposition from established physicists.
5) The author claims this approach shows its worth when calculating the effects of time.
Forgive Savants Their Midwittery
Random things I learnt about ASML after wondering how critical they were to GPU progress.
ASML makes specialized photolithography machines. They’re about a decade ahead of competitors i.e. without ASML machines, you’d be stuck making 10nm chips.
They use 13.5nm “Extreme UV” to make 3nm scale features by using reflective optics to make interference patterns and fringe. Using low res light to make higher res features has been going on since photolithography tech stalled at 28nm for a while. I am convinced this is wizardry.
RE specialization: early photolithography community used to have co-development between companies, technical papers sharing tonnes of details, and little specialization in companies. Person I talked to says they don’t know if this has stopped, but it feels like it has.
In hindsight, no-one in the optics lab at my uni talked about chip manufacturing: it was all quantum dots and lasers. So maybe
It’s unclear how you can decrease wavelength and still use existing technology. Perhaps we’ve got 5 generations left.
We might have to change to deep UV light then.
Even when we reach the limits of shr
ASML makes machines for photolithography, somehow using light with λ > chip feature size. If ASML went out of business, everyone wouldn’t be doomed. Existing machines are made for particular gens, but can be used for “half-steps”. Like from 5nm to 4nm. Everyone is building new fabs, and ASML is building new machines as fast as they can.
Would prob trigger world recession if they stopped producing new things.
Very common in tech for monopoly partners to let customer’s get access to their tech if they go out of business.
TSMC and Intel buys from ASML.
Don’t seem to be trying to screw people over.
If they tried, then someone else would come in. Apple might be able to in like 10 or even twenty years.
China has tried hard to do this.
ASML have edges in some fabs, other companies have edges in different parts of the fab.
Some companies just started specializing more in the sorts of machines they had in the fabs.
Cannon and Nikon make other photolithography machines in fabs, but specialize in different sorts for different purposes.
ASML’s are used in bottom most layers, used for transistors. Other companies focus on higher layers, with “registration requirements being less strict”.
Might still be in the decade range.
If you didn’t have ASML tech, you’d need to fall back to 10nm tech.
Just TSMC at 3nm in production.
Everyone behind them, Intel Samsung, are also ASML customers.
Friend’s company is made using TSMC. They give masks, and get chips made.
Do you just naturally get monopolies in this industry?
Used to have tonnes of info sharing. Technical papers were shared tonnes.
Making things got harder, and people said it was too important not to share.
Worried about China using these things, for kind of spurious reasons (they can already make ICBMS to ruin everyone’s day)
Used to be co-developing between companies.
Don’t know why that stopped. Or even if it really has, it just feels like it has stopped.
Very little discussion of chip manufacturing in hindsight.
Extreme UV is like 12nm light (much shorter than prior ~100nm), won’t go through glass lenses. Try to use reflective optics as much as possible.
At microprocessors report, Intel was saying they’d make their own machines to do this and would show others how to.
They would do this to show they’d maintain their technical edge.
They said they’d get it done by 2010, and they were saying this in like 1995?.
Ended up taking twice as long. Only started getting it in 7nm.
Don’t know how much we’re relying on ASML vs Intel tech.
Hoping to get EUV working, but took longer, and was hard to use w/o EUV. Intel said it would be ready at 28nm, and it wasn’t, so they had to use lower resolution light to somehow pull it off.
Somehow using fringes diffraction to get higher res.
What are upcoming technologies in the photolithography stuff?
Not sure how much more you can decrease wavelength and still use existing technology.
Maybe 5 generations past where we are without changing anything.
And then might have to change to deep UV.
They’re using 13.5nm light.
Process tech can improve in different ways.
1nm, when introduced, will have low yield. After 10 years, essentially all chips will be made correctly.
Standard experience curve stuff applies.
Eking out all the economic performance of chip making techniques will take like 20 years after you get to the limits of shrinking dies
This would translate directly into continuous improvements in PC’s, AIs and that sort of thing.Lots of hardware optimization has happened, and this is partly a software thing i.e. you make hardware more optimized for some software, and improve the software on chips. Which muddies the algorithm vs hardware split you get.
So IRL super villains will openly monologue on their plans, but when it is convenient they’ll say “I don’t want to set fire to the moon any more, trust me bro”?
IRL they don’t, but in fiction they do.
Supervillain Monologues Are Unrealistic
Centralization begets stagnation
Genius is Not About Genius
Oh, thanks for catching that!
What was the novel?