If you were to transport me and Algon into the wonderful universe of Mother of Learning [...] I’d absolutely mog him. I haven’t even read the book, but now I know for sure I’d mog him.
As I told Croissanthology privately after he wrote this reply, I never said I wouldn’t co-operate with my clone/original.
Fairly active Less wrong users? Probably not <10%, though I wouldn’t be shocked, just surprised.
consequent shame afterwards
Speak for yourself.
The upshot is that, it seems to me, oxytocin is pretty antithetic to ambition. And not just ambition “at the grand scale”; also smaller-scale ambitions, relevant to the whole range of non-oxytocin-driven values.
Or in other words, “Family, religion, friendship. These are the three demons you must slay if you wish to succeed in business.”
What was the novel?
No, and that is a good idea.
Tangentially relevant: Armstrong has a post on some general features that make it likely that Goodharting will be a problem. Conversely, he gives some features that make it less likely you’ll wind up Goodharting in your search.
So, as long as:
We use a Bayesian mix of reward functions rather than a maximum likelihood reward function.
An ideal reward function is present in the space of possible reward functions, and is not penalised in probability.
The different reward functions are normalised.
If our ideal reward functions have diminishing returns, this fact is explicitly included in the learning process.
Then, we shouldn’t unduly fear Goodhart effects (of course, we still need to incorporate as much as possible about our preferences into the AI’s learning process). The second problem, making sure that there are no unusual penalties for ideal reward functions, seems the hardest to ensure.
In the roguelike Angband, the player has to descend 100 floors of the titular dungeon to kill Morgoth and beat the game. Novices hang about the shallower floors, slaying mini-bosses, gaining levels, collecting loot and not descending to further levels. Inevitably, they die well before reaching floor 100. Old hands advise against this and say to beat the game you need to focus on going down. Inevitably, novices do not listen until they’ve died countless times and beaten the game. At which point, the cycle repeats.
All of that is to say: it’s true that you need to play to win. It’s also true that people don’t get this. One exception is people who’ve played to win in the past, usually after many failures as they stubbornly ignore advice on how to win.
I should know, as I was that novice in Angband at one point. Eventually, I gave in and gave in to victory, descending as fast as I could handle. For quite some time, this was difficult. Even though I explicitly aimed to win, for some reason I would eventually just lose the ability to focus on descending and start puttering about collecting loot like a magpie. Rest was required before I could get my head back in the game. If I didn’t, the result was death.
Why exactly my ability to play to win could be sapped so easily confused me. I wasn’t sure if it was because of bad habits, or because of some generalized failure to follow an explicit goal or what. So once I beat Angband, I decided to determine why I struggled to win by playing another roguelike where you win by descending through a dungeon: Noita.
And what do you know, I still struggled to play to win! I still struggled to descend or execute on any strategy like “acquire as much health as possible”, “beat this dude to unlock this spell for future runs” etc. So it wasn’t bad habits that killed me in Angband, but a generalized difficulty in optimizing for a goal for an extended period of time.
Perhaps then playing to win is like a resource you can spend, and most people just don’t care to spend that resource on board games. More optimistically, playing to win is like a muscle you can train. In which case, you want to play to win at board games.
I’m not sure which is true. But either way, everyone should experience of playing to win at least a few times in their lives, till they learn what it feels like and can tell when they’re playing to win, and when they’re not.
Some interesting books from my reading list.
Analytical Mechanics—Nivaldo A Lems
In the tradition of V.I. Arnold, Goldstein, and Landau & Lifshitz. The author is really classical mechniacs pilled. Though the book is more focused on the mathematics of it all than the physics.
Fundamental Principles of Classical Mechanics
In the same tradition as Lems but more heuristic. It uses physical motivation to build up to formal ideas. Places a greater focus on a local co-ordinate approach. The author emphasises Cartan’s Method of Moving Frames, for pedagogical purposes. This is linked to the historical development of the subject, which I quite like. But notably, the author says Goldstein is the gold standard for the mathematical methods of classical mechanics. So why bother reading this book?
Differential Geometry in Physics—Gabriel Lugo
Has coloured figures. La-dee-da.
Pre-reqs: Calculus, Linear Algebra and Differential Equations. Assumes the reader has encountered the concepts in classical/quantum mechanics.
Aims to:
1) Bridge differential geometry’s historical practical formulation and its abstract modern form.
2) Help physicists who want to strengthen their mathematical ability. The author was shocked that HPE folk and differential geometry folk worked on connections on fiber bundles whilst unaware that they were really working on the same stuff. This was part of the inspiration of this book, meant to rectify the situation.
3) Be readable to those at the boundary of maths/physicists.
4) Serve as background for GR.
5) Add material that is unusual for a textbook, such as solitions on the style of Backlund transform instead of Olver.
Geometry from dynamics: classical and quantum.
Looks interesting, certainly a different perspective.
Something about dynamics giving rise to geometric structure e.g. Hamiltonian, Lagrangian and Symplectic geometries.
Claims geometry is experienced first over dynamics.
Relativity and Cosmology—Thorne & Blandford
You know you want it.
Emphasises a geometric approach.
Takes 1/4-1/2 of a semester to teach.
Emphasises historical transformations in the physics of GR.
Has practical calculations!
Structure and Interpretation of Classical Mechanics − 2nd Edition.
1) Uses Scheme. Apparently they don’t need to teach it, as students just learn it in a matter of days.
2) Notation is built to make turning equations into programs easy. The author claims this simplifies things as a result.
3) Focuses on motion over deriving equations of motion.
3) Non-linear dynamics is given a lot of time throughout the text.
4) Lots of time spent on phase space perspective.
5) This book has many cool qualitative results.
Concise Treatise on Quantum Mechanics in Phase Space
1) This book focuses on Wigner’s formulation of quantum mechanics. Wigner’s Quasi-Probability distribution function in phase space furnishes a representation of quantum mechanics on equal footing to the Hamiltonian and Path Integral formulation.
2) The author claims this representation more naturally and intuitively connects to the classical limit.
3) The representation is weird. You have simultaneous position and momentum variables
4) The theory was built by two unknown researchers, and faced opposition from established physicists.
5) The author claims this approach shows its worth when calculating the effects of time.
Have you looked at the historical development of homology? IME that’s generally a good way to build intuitions on a topic, as it tends to make concepts appear natural.