I’m confused by this response, so let’s some numbers on it.
Suppose you have enough compute to train a model with 2 trillion parameters with the conventional backpropagation algorithm. If you had a better algorithm that didn’t incur the memory overhead of backprop with its global update rules, you could use the same hardware to train a model triple the size, which is a 6T model.
Scaling laws tell us that we can reasonably expect this to be a much more capable model.
Possibly controversial, but I think the biggest thing that is wrong with modern deep learning is that backpropagation is the wrong learning rule.
Reading Reiner Pope’s “How to Scale Your Model”, backpropagation triples compute cost compared to inference, which means that it is not economically feasible to deploy large models that learn online.
This is absurd! This cannot be the Master Learning Algorithm that the human brain uses to implement AGI at 20W power consumption.
I recently heard Ilya Sutskever say that his heuristic is to draw inspiration from the best understanding of how the human brain works, and use that as a “good taste“ heuristic as to what is likely to work. In this context, backpropagation is terrible taste, absolutely disgusting.
The next iteration of learning updates will most likely be lighter. Probably a modernization of Hebbian learning.
As a lapsed violinist, I find this incredibly insightful. Explains a lot about the tendencies I see in my personality in other contexts as well.
Thanks for writing it!
I think this is an amazing question to predict on. Can you please try to get it on a platform? personally I like Metaculus, but anything is fine
I like chess :) it’s an eternal classic for a reason.
It teaches thoroughness in thinking through the space of possibilities, as well as a healthy aggression. The game is biased towards offense, as is life. Also tenacity under pressure—holding on when you feel lost, and being rewarded with a blunder from an overconfident opponent.
I like to play correspondence on lichess, 24 hours per move. Once (or twice) a day, I pull up the app, and think through my next move—really think, chess punishes rash decision-making.
I do feel it translates to how I approach decisions in real life.
Great and thought provoking article! I immediately thought of “spread spectrum techniques”. In contrast with radio transmission techniques that use a single communication frequency, “spread spectrum” techniques spread the signal across a wide array of frequencies.
Somebody trying to intercept the communication may not even be able to distinguish the signal from background noise. The only way to detect it is to know how to invert the spreading function.
https://en.wikipedia.org/wiki/Spread_spectrum
In general I take issue with terms like “sum-max”, the inverse of the spreading function might not be a sum, it could be arbitrary and nonlinear. I bet your neural network examples are nonlinear.
I think one thing you can do is get involved in spaces and activities in which it is known that effort and diligence give results.
The classic example is exercise. Other examples could be tidying your home, the author gives the example of cooking for yourself.
Personally, during Covid I was one of those people who got really into chess. I did a LOT of practice puzzles and developed a sharp tactical instinct, as well as a sense of gritty perseverance and attention to detail. More importantly, I do feel it improved my sense that “I can get better at things, look, that’s the graph of my chess rating”.
I think it makes sense to find areas of life where feedback is faster, improve morale, and port that over to careers and relationships, where feedback is slower and often full of setbacks. You might get an amazing job, only to find that your company does layoffs, and you struggle to find a similar job and have to take a step back into less interesting work for a time (I am personally in the middle of this, lol)
I’m confused—your examples for IVT reference “lumpy” functions. I’m not exactly sure what that means, but it seems like you mean functions with discrete, sharp steps. Such a function would be discontinuous, and IVT only applies to continuous functions.
Actually I wonder if we could do an experiment in the following way:
Collect Wikipedia in English
Collect Wikipedia in some other language, e.g. Japanese
Train an LLM on these two languages
Try it out!
It’s true that there will be some amount of overlap, but this should put a ceiling on how well this approach could work.
I guess it’s about entropy. Some items have more entropy than others. They have more parts and more possible states, and most of those states are misconfigured.
A table or a chair has most of its parts glued together in a fixed, low-entropy position, and has few parts to begin with.
A printer has many parts, both at the hardware and software level, and they can be in many states. It has paper, which can be in a state of having run out or not, and if it has run out, the supply can be almost literally anywhere. It has many intricate mechanical parts which can wear out at different rates, it has an ink cartridge that can dry out even when unused, and many intricate little parts that I don’t even know about.
On top of that, the software also has high complexity. Lots of software components, including the low level stuff that manages the actual printing, whatever needs to connect to the wifi, whatever is needed in case it can print from email, etc. all of these processes can run concurrently, which may introduce race conditions, any of these components may have a bug.
And then there’s the drivers installed on your laptop, which may be out of date, or your laptop’s drivers may be fully up to date but the printer software is out of date and your driver is no longer compatible with it.
Many components, many possible states, high entropy.
EDIT: also, I think a big part of it is that printers aren’t really tested for reliability. I used to work at a car manufacturer. Cars are hybrid software-hardware systems, and both sides of that equation are highly complex. Car engineering is a huge discipline that focuses aggressively on reliability. Even though car makers appear to come out with a new model every year, in reality they only change the cosmetics. The underlying platform usually only changes once every six years, and every version of the platform is aggressively stress tested in every possible way. The company I worked for had a test track in the coldest possible part of Canada, as well as in Death Valley, because reliability testing has to be aggressive and thorough.
Also the software side used special, hardened compilers that only supported a subset of the programming language, and were rarely changed. Getting a contract to supply a car company with a critical tool like a compiler is a hugely profitable multi-year deal, because they are happy to pay for reliability and reluctant to change something that is working well.
I bet nobody really cares about the reliability of printers to the same extent, and they have huh entropy, so they generally suck.
Where can one buy ostrich eggs?
This comic by Tim Urban is interesting, but I remember when I first read it, it seemed wrong.
In his framework, I think ASI can only be quantitatively more powerful than human intelligence, not qualitatively.
The reason is simple: humans are already Turing complete. Anything a machine can do, it can only be faster execution of something a human could already do.
I don’t think it has much bearing on the wider discussion of AI/AI-risk, I haven’t heard anybody else think that the distinction of quantitative/qualitative superiority had any bearing on AI risk.
I have similar problems, and I was hoping you had found a solution. One question I have: on a typical night, do you have good sleep quality, for example as measured by a fitness tracker?
Closest thing I have found to a solution of daily energy is creatine, 5g daily, and 10g if I did hard exercise that day. I cycle on it for about two weeks, until it starts to feel like too much of a good thing, then cycle off for a week, repeat.
Super interesting! I have some follow-up thoughts to this.
On the one hand, it seems that this is a case of “having” more working memory.
On the other hand, it might be a case of more experience allowing you to filter out irrelevant things and, allowing more working memory to focus on the things that matter.
A classical example of the latter point is when novices learn to play chess. In a typical training session, they will be presented with a position, and be asked to propose a move. They will take a moment, then propose a move that loses a queen in a totally obvious way, and when challenged, they will take a moment and confirm that “oh, yeah, that loses a queen”. Typically a novice is overwhelmed by the details of the position, perhaps still remembering how the pieces move, and have no sense of which moves are important and which moves should be ignored. As a result, they can’t do “basic” one-step lookahead to see what countermoves the opponent has. Their working memory is too overloaded with all of the possibilities on the board.
This gets resolved with practice, drills, and experience. Eventually the hind-brain simply “surfaces” the information “that square is controlled by an enemy bishop”, and in many cases will not even suggest the move that would hazard the queen, it won’t even get loaded into conscious attention.
EDIT: Building on this, it’s super interesting to watch Grandmasters like Hikaru Nakamura play games live on stream, and comment on their thought process. Yes, they calculate long lines, sometimes branching, but as they talk, partially distracted, they never consider bad moves, they’re always analyzing the top moves recommended by the computer.
I love this post, and I have been thinking a lot about it since it came out.
One counter argument I can think of is the following.
The wizard capability of a single human is actually not very high
Human wizardry achieves greatest power when many wizards join forces and coordinate at scale
Hence, the most powerful people are those who can coordinate and direct large groups of wizards, in the language of this article, “kings”
So one may argue that it’s rational to seek king-power on this basis
Despite this, I personally resonate very strongly with the image of the wizard, and I think the world would be a better place if we all sought to strengthen our wizarding powers.
Can you say more about what kind of changes you implemented?
A perhaps more interesting interaction is with wills that are managed by trusts. My understanding is that you can put conditions on how the money in a trust will be disbursed to your heirs, for example “as long as they maintain a minimum GPA in college”. I have heard lawyers make outrageous jokes like adding a clause that says “as long as they don’t marry that person”.
It’s quite reasonable to expect that some will add a clause to their trust that says “this only pays out if they have placed themselves on the lifetime no-gambling list”.
a google search suggests desoxyn might be just be a brand of pharmaceutical-grade meth
Would you mind publishing the protocol?
An interesting idea, but I notice that:
Professional athletes do a significant amount of conventional strength training.
Professional musicians practice scales, arpeggios, and pieces called “studies” in addition to their concert pieces, and even do hand mobility exercises. “Studies” are usually short, boring pieces that challenge the limits of dexterity.
These conventional, supplementary strategies push capabilities beyond those required at the time of performance, ensuring that the performer has spare capacity, not operating at their limits when it counts. Also, many exercises have a complementary nature, such as athletes strengthening muscles that balance their physique and prevent injuries.
Overall, by observing the time-tested training methods of high-functioning professionals from in fiercely competitive fields, I think the advice in this post is mostly incorrect :)
High-functioning professionals don’t learn to restrain bulls by actually trying to restrain a bull on day one, that’s how you get gored. You look at the task, decompose it into subskills, drill the subskills until you can do them better than you will need to in the real task, then practice stringing them together, and then you go restrain bulls and catch lobsters.