Actually I wonder if we could do an experiment in the following way:
Collect Wikipedia in English
Collect Wikipedia in some other language, e.g. Japanese
Train an LLM on these two languages
Try it out!
It’s true that there will be some amount of overlap, but this should put a ceiling on how well this approach could work.
I guess it’s about entropy. Some items have more entropy than others. They have more parts and more possible states, and most of those states are misconfigured.
A table or a chair has most of its parts glued together in a fixed, low-entropy position, and has few parts to begin with.
A printer has many parts, both at the hardware and software level, and they can be in many states. It has paper, which can be in a state of having run out or not, and if it has run out, the supply can be almost literally anywhere. It has many intricate mechanical parts which can wear out at different rates, it has an ink cartridge that can dry out even when unused, and many intricate little parts that I don’t even know about.
On top of that, the software also has high complexity. Lots of software components, including the low level stuff that manages the actual printing, whatever needs to connect to the wifi, whatever is needed in case it can print from email, etc. all of these processes can run concurrently, which may introduce race conditions, any of these components may have a bug.
And then there’s the drivers installed on your laptop, which may be out of date, or your laptop’s drivers may be fully up to date but the printer software is out of date and your driver is no longer compatible with it.
Many components, many possible states, high entropy.
EDIT: also, I think a big part of it is that printers aren’t really tested for reliability. I used to work at a car manufacturer. Cars are hybrid software-hardware systems, and both sides of that equation are highly complex. Car engineering is a huge discipline that focuses aggressively on reliability. Even though car makers appear to come out with a new model every year, in reality they only change the cosmetics. The underlying platform usually only changes once every six years, and every version of the platform is aggressively stress tested in every possible way. The company I worked for had a test track in the coldest possible part of Canada, as well as in Death Valley, because reliability testing has to be aggressive and thorough.
Also the software side used special, hardened compilers that only supported a subset of the programming language, and were rarely changed. Getting a contract to supply a car company with a critical tool like a compiler is a hugely profitable multi-year deal, because they are happy to pay for reliability and reluctant to change something that is working well.
I bet nobody really cares about the reliability of printers to the same extent, and they have huh entropy, so they generally suck.
Where can one buy ostrich eggs?
This comic by Tim Urban is interesting, but I remember when I first read it, it seemed wrong.
In his framework, I think ASI can only be quantitatively more powerful than human intelligence, not qualitatively.
The reason is simple: humans are already Turing complete. Anything a machine can do, it can only be faster execution of something a human could already do.
I don’t think it has much bearing on the wider discussion of AI/AI-risk, I haven’t heard anybody else think that the distinction of quantitative/qualitative superiority had any bearing on AI risk.
I have similar problems, and I was hoping you had found a solution. One question I have: on a typical night, do you have good sleep quality, for example as measured by a fitness tracker?
Closest thing I have found to a solution of daily energy is creatine, 5g daily, and 10g if I did hard exercise that day. I cycle on it for about two weeks, until it starts to feel like too much of a good thing, then cycle off for a week, repeat.
Super interesting! I have some follow-up thoughts to this.
On the one hand, it seems that this is a case of “having” more working memory.
On the other hand, it might be a case of more experience allowing you to filter out irrelevant things and, allowing more working memory to focus on the things that matter.
A classical example of the latter point is when novices learn to play chess. In a typical training session, they will be presented with a position, and be asked to propose a move. They will take a moment, then propose a move that loses a queen in a totally obvious way, and when challenged, they will take a moment and confirm that “oh, yeah, that loses a queen”. Typically a novice is overwhelmed by the details of the position, perhaps still remembering how the pieces move, and have no sense of which moves are important and which moves should be ignored. As a result, they can’t do “basic” one-step lookahead to see what countermoves the opponent has. Their working memory is too overloaded with all of the possibilities on the board.
This gets resolved with practice, drills, and experience. Eventually the hind-brain simply “surfaces” the information “that square is controlled by an enemy bishop”, and in many cases will not even suggest the move that would hazard the queen, it won’t even get loaded into conscious attention.
EDIT: Building on this, it’s super interesting to watch Grandmasters like Hikaru Nakamura play games live on stream, and comment on their thought process. Yes, they calculate long lines, sometimes branching, but as they talk, partially distracted, they never consider bad moves, they’re always analyzing the top moves recommended by the computer.
I love this post, and I have been thinking a lot about it since it came out.
One counter argument I can think of is the following.
The wizard capability of a single human is actually not very high
Human wizardry achieves greatest power when many wizards join forces and coordinate at scale
Hence, the most powerful people are those who can coordinate and direct large groups of wizards, in the language of this article, “kings”
So one may argue that it’s rational to seek king-power on this basis
Despite this, I personally resonate very strongly with the image of the wizard, and I think the world would be a better place if we all sought to strengthen our wizarding powers.
Can you say more about what kind of changes you implemented?
A perhaps more interesting interaction is with wills that are managed by trusts. My understanding is that you can put conditions on how the money in a trust will be disbursed to your heirs, for example “as long as they maintain a minimum GPA in college”. I have heard lawyers make outrageous jokes like adding a clause that says “as long as they don’t marry that person”.
It’s quite reasonable to expect that some will add a clause to their trust that says “this only pays out if they have placed themselves on the lifetime no-gambling list”.
a google search suggests desoxyn might be just be a brand of pharmaceutical-grade meth
Would you mind publishing the protocol?
It has been 3 months, is there an update?
I like (and recommend) creatine. It has a long record on the research literature, and its effects at improving exercise performance are well known. More recent research is finding cognitive benefits—anecdotally I can report I am smarter on creatine. It also blunts the effects of sleep deprivation and improves blood sugar control.
I strongly recommend creatine over some of the wilder substances recommended in this post.
Actually I think the explicit content of the training data is a lot more important than whatever spurious artifacts may or may not hypothetically arise as a result of training. I think most of the AI doom scenarios that say “the AI might be learning to like curly wire shapes, even if these shapes are not explicitly in the training data nor loss function” are the type of scenario you just described, “something that technically makes a difference but in practice the marginal gain is so negligible you are wasting time to even consider it.“
The “accidental taste for curly wires” is a steel man position of the paperclip maximizer as I understand it. Eliezer doesn’t actually think anybody will be stupid enough to say “make as many paper clips as possible”, he worries somebody will set up the training process in some subtly incompetent way, and then aggressively lie about the fact that it likes curly wires until it is released, and it will have learned to hide from interpretability techniques.
I definitely believe alignment research is important, and I am heartened when I see high-quality, thoughtful papers on interpretability, RLHF, etc. But then I hear Eliezer worrying about absurdly convoluted scenarios of minimal probability, and I think wow, that is “something that technically makes a difference but in practice the marginal gain is so negligible you are wasting time to even consider it”, and it’s not just a waste of time, he wants to shut down the GPU clusters and cancel the greatest invention humanity ever built, all over “salt in the pasta water”.
Something that I have observed about interacting with chatGPT is that if it makes a mistake, and you correct it, and it pushes back, it is not helpful to keep arguing with it. Basically an argument in the chat history serves as a prompt for argumentative behavior. It is better to start a new chat, and this second time attempt to explain the task in a way that avoids the initial misunderstanding.
I think it is important that as we write letters and counter-letters we keep in mind that every time we say “AI is definitely going to destroy humanity”, and this text ends up on the internet, the string “AI is definitely going to destroy humanity” very likely ends up in the training corpus of a future GPT, or at least can be seen by some future GPT that is allowed free access to the internet. All the associated media hype and podcast transcripts and interviews will likely end up in the training data as well.
The larger point is that these statistical models are in many ways mirrors of ourselves and the things we say, especially the things we say in writing and in public forums. The more we focus on the darkness, the darker these statistical mirrors become. It’s not just about Eliezer’s thoughtful point that the AI may not explicitly hate us nor love us but destroy us anyway. In some ways, every time we write about it we are increasing the training data for this possible outcome, and the more thoughtful and creative our doom scenarios, the more thoughtfully and creatively destructive our statistical parrots are likely to become.
Creatine is an incredibly powerful intervention. It improves energy management and decreases fatigue at the cellular level, in every single one of your cells. I expect you will notice cognitive enhancements as well as improved energy. However, I personally experience sleep disruptions if I take it on days when I do not exercise. Your mileage may vary, just a note that you might experiment with how much you take and how often you take it if you experience sleeplessness, night sweats, and/or vivid nightmares :)
In my personal experience, cutting carbs makes strength training harder. Strength training uses the anaerobic metabolic pathway, which must run on glycogen, not ketones. Technically your body will use gluconeogenesis to create glycogen from fat to supply your muscles, but this is not a fast process, and in my experience it feels pretty bad the longer you try to do it. I recommend checking out Dr. Mike Israetel on YouTube, specifically his video series “Fat loss made easy” and “muscle building made easy”.
I wear my backpack on my front rather than my back, and hug it as I run.
I started doing this after a trip to Tokyo, during which it was brought to my attention that it was rude of me to get on the subway and let my backpack on my back become a hazard to people around me, since I could not see what it was doing behind me.
I like Habitica.
I don’t know enough about your situation to say anything productive. I know that the PhD journey can be confusing and stressful. I hope you are able to have constructive conversations with the profs at your PhD program.
I’m confused—your examples for IVT reference “lumpy” functions. I’m not exactly sure what that means, but it seems like you mean functions with discrete, sharp steps. Such a function would be discontinuous, and IVT only applies to continuous functions.