My theory on why AI isn’t creative is that it lacks a ‘rumination mode’. Ideas can sit and passively connect in our minds for free. This is cool and valuable. LLMs don’t have that luxury. Non-linear, non-goal-driven thinking is expensive and not effective yet.
Yes. See Google’s Co-Scientist project for an example of an AI scaffolded to have a rumination mode. It is claimed to have matched the theory creation of top labs in two areas of science.
So this rumination mode is probably expensive and only claimed to be effective in the one domain it was engineered for. So far. Based on the scaffolded sort-of-evolutionary “algorithm” they used to recombine and test hypotheses against published empirical results, I’d expect that a general version would work almost as well across domains, once somebody puts effort and some inference money into making it work.
This is cool and valuable, as you say. It’s also extremely dangerous, since this lack is one of the few gaps between current LLMs and the general reasoning abilities of humans—without human ethics and human limitations.
Caveat—I haven’t closely checked the credibility of the co-scientist breakthrough story. I think it’s unlikely to be entirely fake or overstated based on the source, but draw your own conclusions.
I’ve primarily thus far taken my conclusions from this podcast interview with the creators and a deep research report based largely on this paper on the co-scientist project.
Looks like Nathan Labenz, the host of that podcast (and an AI expert in his own right) estimates the inference cost for one cutting-edge hypothesis at $100-1000 for one cutting-edge inference based on the literature in this followup episode (which I do not recommend since it’s focused on the actual biological science)
Suppose a model learns “A->B” and “B->C” as separate facts. These get stored in the weights, probably somewhere across the feedforward layers. They can’t be combined unless both facts are loaded into the residual stream/token stream at the same time, which might not happen. And even if that is the case, the model won’t remember “A->C” as a standalone fact in the future, it has to re-compute it every time.
Sure. But more than the immediate, associative leaps, I think I’m interested in their ability to sample concepts across very different domains and find connections whether that is done deliberately or randomly. Though with humans, the ideas that plague our subconscious are tied to our persistent, internal questions.
The aspect of your work to care about the most is replay value. How many times do people keep coming back? Number of replays, rereads, and repeat purchases are proxies for high resonance. On that note, I wish more writing platforms let you see in aggregate how many first-time readers visited again and how spaced out their visits were. If they can still look past the known plot and imperfections in your work, you’re on to something.
i think this is a reasonable proxy for some stuff people generally care about, but definitely faulty as a north star.
some negative examples:
gambling, alcohol, anything addictive
local optima (e.g. your existing userbase would like your product less if you made X change, but you would reach way more people/reach a different set of people and help them more/etc if you made X change)
some products don’t make sense to have repeat customers, e.g. life insurance policies
1 and 3 are not the kind of work I had in mind when writing this take. I see your second point, but I’d want to counter with the fact that what got you from Level 1 to Level 2 won’t be the same thing as what gets you to Level 3 (this is the natural cost of scale). You may outgrow some initial users, but this can be compensated by a low overall churn. Most won’t leave unless your core offering has drastically pivoted.
Work developed through artistic value and/or subjectivity (songs, books, movies, speeches, paintings, consumer products). My point is the greatest works stand the test of time and are typically studied/appreciated over the years. Ex’s like Paul Graham’s essays, albums from decades before, or even the minimalistic design of Apple. If people keep coming back, it got something right and was likely ahead of its time. Compared to other metrics (total impressions or number of comments), repeat behavior tells a less noisy story about the quality of the work. Level, 1, 2,3 were arbitrarily chosen. What I meant was when you move from something early-stage to mainstream, you have to let go of some of the beliefs or ideas that may have garnered your first fans.
I agree with your negative examples, but those are hardly the kind of businesses I’d ever want to work in.
Work developed through artistic value and/or subjectivity
thanks for clarifying! so, to be clear, is the claim you’re making that: work that has artistic or otherwise subjective aims/values can find a measurement of its value in the extent to which its “customers” (which might include e.g. “appreciators of its art” or “lovers of its beauty”) keep coming back.
does that sound like an accurate description of the view you’re endorsing, or am i getting something wrong in there?
Young people would benefit a lot more if they defaulted to forming and defending an opinion in real-time. I would rather say what I think and find out how wrong I am than keep waiting for more data.
This thought came to me during a walk to class. A good professor of mine would show us a graph with a blurred out title. He’d ask for our initial observations or what we think it represents. Even that is intimidating for most because no one wants to say something stupid or too simple. This idea that you can’t have conviction and update your beliefs later needs to be unlearnt.
My theory on why AI isn’t creative is that it lacks a ‘rumination mode’. Ideas can sit and passively connect in our minds for free. This is cool and valuable. LLMs don’t have that luxury. Non-linear, non-goal-driven thinking is expensive and not effective yet.
Cross-posted from X
Yes. See Google’s Co-Scientist project for an example of an AI scaffolded to have a rumination mode. It is claimed to have matched the theory creation of top labs in two areas of science.
So this rumination mode is probably expensive and only claimed to be effective in the one domain it was engineered for. So far. Based on the scaffolded sort-of-evolutionary “algorithm” they used to recombine and test hypotheses against published empirical results, I’d expect that a general version would work almost as well across domains, once somebody puts effort and some inference money into making it work.
This is cool and valuable, as you say. It’s also extremely dangerous, since this lack is one of the few gaps between current LLMs and the general reasoning abilities of humans—without human ethics and human limitations.
Caveat—I haven’t closely checked the credibility of the co-scientist breakthrough story. I think it’s unlikely to be entirely fake or overstated based on the source, but draw your own conclusions.
I’ve primarily thus far taken my conclusions from this podcast interview with the creators and a deep research report based largely on this paper on the co-scientist project.
Looks like Nathan Labenz, the host of that podcast (and an AI expert in his own right) estimates the inference cost for one cutting-edge hypothesis at $100-1000 for one cutting-edge inference based on the literature in this followup episode (which I do not recommend since it’s focused on the actual biological science)
Do you mean something like:
Suppose a model learns “A->B” and “B->C” as separate facts. These get stored in the weights, probably somewhere across the feedforward layers. They can’t be combined unless both facts are loaded into the residual stream/token stream at the same time, which might not happen. And even if that is the case, the model won’t remember “A->C” as a standalone fact in the future, it has to re-compute it every time.
Sure. But more than the immediate, associative leaps, I think I’m interested in their ability to sample concepts across very different domains and find connections whether that is done deliberately or randomly. Though with humans, the ideas that plague our subconscious are tied to our persistent, internal questions.
Gwern’s made some suggestions along similar lines.
The aspect of your work to care about the most is replay value. How many times do people keep coming back? Number of replays, rereads, and repeat purchases are proxies for high resonance. On that note, I wish more writing platforms let you see in aggregate how many first-time readers visited again and how spaced out their visits were. If they can still look past the known plot and imperfections in your work, you’re on to something.
i think this is a reasonable proxy for some stuff people generally care about, but definitely faulty as a north star.
some negative examples:
gambling, alcohol, anything addictive
local optima (e.g. your existing userbase would like your product less if you made X change, but you would reach way more people/reach a different set of people and help them more/etc if you made X change)
some products don’t make sense to have repeat customers, e.g. life insurance policies
1 and 3 are not the kind of work I had in mind when writing this take. I see your second point, but I’d want to counter with the fact that what got you from Level 1 to Level 2 won’t be the same thing as what gets you to Level 3 (this is the natural cost of scale). You may outgrow some initial users, but this can be compensated by a low overall churn. Most won’t leave unless your core offering has drastically pivoted.
what kind of work did you have in mind when writing this take?
what do you mean by Levels 1, 2, or 3? i have no idea what this is in reference to.
Work developed through artistic value and/or subjectivity (songs, books, movies, speeches, paintings, consumer products). My point is the greatest works stand the test of time and are typically studied/appreciated over the years. Ex’s like Paul Graham’s essays, albums from decades before, or even the minimalistic design of Apple. If people keep coming back, it got something right and was likely ahead of its time. Compared to other metrics (total impressions or number of comments), repeat behavior tells a less noisy story about the quality of the work. Level, 1, 2,3 were arbitrarily chosen. What I meant was when you move from something early-stage to mainstream, you have to let go of some of the beliefs or ideas that may have garnered your first fans.
I agree with your negative examples, but those are hardly the kind of businesses I’d ever want to work in.
thanks for clarifying! so, to be clear, is the claim you’re making that: work that has artistic or otherwise subjective aims/values can find a measurement of its value in the extent to which its “customers” (which might include e.g. “appreciators of its art” or “lovers of its beauty”) keep coming back.
does that sound like an accurate description of the view you’re endorsing, or am i getting something wrong in there?
yes that’s basically it, thanks!
Young people would benefit a lot more if they defaulted to forming and defending an opinion in real-time. I would rather say what I think and find out how wrong I am than keep waiting for more data.
This thought came to me during a walk to class. A good professor of mine would show us a graph with a blurred out title. He’d ask for our initial observations or what we think it represents. Even that is intimidating for most because no one wants to say something stupid or too simple. This idea that you can’t have conviction and update your beliefs later needs to be unlearnt.