Dreams of Ideas
CGP Grey describes a phenomenon I think of as ‘dreams of ideas’ that I find useful as a tool to know when to stop working on a project.
You can be working on something and you are thinking about what it could be, but what is hard to know is if that thought in your head is more like when your brain dreams.
There is a phenomenon that I always think of as like, you wake up and you think you had a great idea in your dream, but what you really had was the idea of a great idea. You didn’t have a great idea, but you had all the emotional resonance to it being a good idea in your dream. You’re like, “Oh, why can’t I think of the details?”. It’s like, because there weren’t any details there at all...
Sometimes there just isn’t the better version of the thing. Like, that’s not real. You can’t get the details right because those details don’t exist. You’re having some kind of different response to the thing that you’re working on, which is, “This thing isn’t good. I can imagine a better version of it.” But can you really? Because, like, the details are what matter.
I sometimes get an idea for a good story and feel inspired to try writing some fiction. Then I sit down to write and realize that I don’t really know what my idea was beyond a vague image of one tiny slice of a story.
Sometimes I think of awesome research ideas and feel excited to start a project. Then I sit down to start coding and realize that I can’t implement it because my idea wasn’t really complete or coherent. I had a vibe of the sort of the thing I’d like to have done, but not an actual plan for what to do.
When to cull ideas
This sort of sounds like the opposite advice to Butterfly Ideas but it’s actually not in contradiction. When an idea is first born, it should be nurtured and protected from excessive rigor. But it’s not healthy for it to remain in that state for too long. You need to set a 5 minute timer and think hard about the thing to see if you can find the details to make it work. Later in the lifecycle, once it’s had time to develop, if an idea hasn’t matured, then it’s time to start asking if this is a real idea or just a mirage.
Examples
I think it’s particularly useful to think about this very late in the lifecycle of an idea. Once dozens of people have thought about something for months or years, then there should be very clear details in place, or very good excuses for why the idea isn’t concrete yet.
It’s obviously problematic to characterize a whole academic discipline with a broad and dismissive brush. And I do genuinely think there are good and concrete ideas to be found there. But the study of complex systems seems to some extent to be a dream of an idea. There is a lot of appeal around the vagaries of explaining emergent phenomena but the technical methods employed are often underwhelming compared to this vision. For example, simulations, especially simulations of many individual agents (LLM or otherwise) are exciting but have delivered relatively few important discoveries.
B2B companies are particularly vulnerable to becoming enamored with dreams of ideas because they only need to sell the dream to another company to make money (at least at first). A software company I once worked for hired a new team to build a ‘knowledge graph’ product. Whenever I spoke to the team I was always left a little unsatisfied by their answer to the question “what is this product and how is it useful?” A knowledge graph is basically a type of a database. And sure, databases are useful, but somehow ‘Knowledge Graph’ evokes much greater expectations, even in the people making them.
Early stage startups will also often maintain dreams of ideas for too long. Many AI startups have big dreams but soon realize that the best product they can make is a simple LLM API wrapper. Sometimes that creates exciting new value, but other times it ends up being less useful than the LLM it is wrapping.
Policy documents often include calls for ‘innovation’. Of course innovation is a real thing and we should make policies that enable it. But a call for innovation itself is almost the Platonic ideal of a dream of an idea. It’s the idea to have better ideas. Which is generally not a very useful idea.
Not every unfulfilled idea is a dream
Mechanistic interpretability has had few big wins, but I don’t think it’s a dream of an idea. To me, reverse engineering a neural network means being able to (at least in theory) hand write a normal human understandable computer program which performs the same function as the network. And there are concrete techniques like SAEs or Activating Patching that people have tried to apply to this problem.
The unified theory of physics is yet to be found, but may well be some day. The point being that, just because an idea hasn’t been fully solved, doesn’t mean it will always be a dream.
The solution
Luckily it’s quite straightforward to mitigate this thought disease in your own life. Any time you have an exciting idea, write it down as quickly as possible, including all the detail. (This is good for many other reasons as well—like not forgetting your ideas). The back of your mind is the warm hospitable environment in which soothing dreams of ideas can fester and steal resources from real, useful thoughts. By exorcising the thought out of your daydreams and down onto paper, you can quickly expose it as vacuous hopium if that’s all it is.
This blog post, on the other hand, was the result of quickly writing down a thought and deciding that it was actually worthwhile.
TL;DR
A “dream of an idea” is an idea that you think is exciting or valuable but lacks the concrete details that make a “real idea” useful. People often dream of ideas for too long because it is more comfortable than facing the harsh reality of implementation.
I think this post is pointing at something very real, but doesn’t quite cleave reality at the joints...
Take “I have the idea of a story”, where “the idea” is “a vague image of one tiny scene in a story”. If you actually properly zoom-in on that image, tons of concrete high-level details about the actual story would likely fall out of it. There would be the object-level features of the scene (what characters are present, what their narrative roles are); the assumptions sitting at the back of your mind regarding how these details should be interpreted (what genre conventions have been established, what’s the overall context in which this scene makes sense); the implicit vibe you want the readers to have when reading that scene (what emotional state they should be in, what they should expect to happen next, whether they should be positively/negatively disposed towards specific characters); so on.
When you start from a vague image, these details aren’t immediately available in the form of declarative knowledge. You basically need to solve the inverse vibing problem: answer the question of “what concrete details would result in this vibe I’m aiming for?”. But solving that problem is very much possible, and gets easier with practice. Afterwards, you’d end up with a general idea of the story structure that could embed/generate your initial vague image.
Similar for “let’s build a startup that does X”, or “let’s build a theory of complex systems”, or “let’s solve alignment via Y”. You may be starting from indirect desiderata – e. g., target observables, or semi-metaphorical plans/vibes – but those still function as constraints. If you approach it correctly, you’d still be able to propagate those constraints around your world-model, all the way to (constraints over) “what is it I’m building?” or “what motor actions I should take next?”.
So the issue isn’t really that the “dreams of ideas” don’t have details. They do, just in an implicit/compressed form.
My take on a similar concept is: Suppose we have some goal, such as “write an interesting story”, or “make a million-dollar startup”, or “solve alignment”, or “incentivize innovation”. An “empty” idea is an idea that seems like it solves the hard parts of achieving this goal, but actually doesn’t. For example,
A vague image of a scene which, when you “unfold” it into the story (via the above approach), results in a story you don’t find interesting at all.
A startup idea which, when you lock down the implementational details, just reduces to “an LLM wrapper”, not capable of attracting any investment.
An alignment scheme which turns out not to solve any of the core difficulties at all: under it, they’re still open questions answering which is where the real work is.
An innovation-incentivizing plan which amounts to “put calls for innovation into a policy document”, which will obviously not actually result in tons of innovations.
In the constraint-propagation interpretation: “empty ideas” are ideas that, once you condition your plans on implementing them, fail to prune the implementational possibility-space to only those constructions that achieve your goal. After the conditioning, you still have to figure out how to make the story interesting, how to attract investors/users, how to solve the hard problems, how to incentivize innovation.
The difference from “dream ideas” is that those ideas are very much possible to translate into implementational details. It’s just that those details are unhelpful for solving the problem you care about.
I think the reason why dreams of ideas are generally not that useful is because they are usually “empty”. The specific details of ideas are so important that if you haven’t actually figured out what those details are, then your prior should be that, when you try to flesh it out, the idea is probably not going to be much better than what you would come up with 5 seconds of wishful thinking.
Seems plausible, yep!