My view on this is that writing a worthwhile blog post is not only a writing task, but also an original seeing task. You first have to go and find something out in the world and learn about it before you can write about it. So the obstacle is not necessarily reasoning (“look at this weird rock I found” doesn’t involve much reasoning, but could make a good blog post), but a lack of things to say.
There’s not any principled reason why an AI system, even a LLM in particular, couldn’t do this. There is plenty going on in the world to go and find out, even if you’re stuck in the internet. (And even without an internet connection, you can try and explore the world of math.) But it seems like currently the bottleneck is that LLM’s don’t have anything to say.
Maybe novels might require less of this than blog posts, but I’d guess that writing a good novel is also a task that requires a lot of original seeing.
I feel like I’ve heard this before, and can sympathize, but I’m skeptical.
I feel like this prescribes an almost magical thinking to how many blog posts are produced. The phrase “original seeing” sounds much more profound than I’m comfortable with for such a discussion.
Let’s go through some examples:
Lots of Zvi’s posts are summaries of content, done in a ways that’s fairly formulaic.
A lot of Scott Alexander’s posts read to me like, “Here’s an interesting area that blog readers like but haven’t investigated much. I read a few things about it, and have some takes that make a lot of sense upon some level of reflection.”
A lot of my own posts seem like things that wouldn’t be too hard to come up with some search process to create.
Broadly, I think that “coming up with bold new ideas” gets too much attention, and more basic things like “doing lengthy research” or “explaining to people the next incremental set of information that they would be comfortable with, in a way that’s very well expressed” gets too little.
I expect that future AI systems will get good at going from a long list of [hypotheses of what might make for interesting topics] and [some great areas, where a bit of research provides surprising insights] and similar. We don’t really have this yet, but it seems doable to me.
We probably don’t disagree that much. What “original seeing” means is just going and investigating things you’re interested in. So doing lengthy research is actually a much more central example of this than coming up with a bold new idea is.
As I say above: “There’s not any principled reason why an AI system, even a LLM in particular, couldn’t do this.”
I think some of it is that I find the term “original seeing” to be off-putting. I’m not sure if I got the point of the corresponding blog post.
In general, going forward, I’d recommend people try to be very precise on what they mean here. I’m suspicious that “original seeing” will mean different things to different people. I’d expect that trying to more precisely clarify what tasks or skills involved would make it easier to pinpoint which parts of it are good/bad for LLMs.
doing lengthy research and summarizing it is important work but not typically what I associate with “blogging”. But I think pulling that together into an attractive product uses much the same cognitive skills as producing original seeing. The missing step in the process you describe is figuring out when the research did produce surprising insights, which might be a class of novel problems (unless a general formulaic approach works and someone scaffolds that in). To the extent it doesn’t require solving novel problems, I think it’s predictably easier than quality blogging that doesn’t rely on research for the novel insights.
“The missing step in the process you describe is figuring out when the research did produce surprising insights, which might be a class of novel problems (unless a general formulaic approach works and someone scaffolds that in).”
-> I feel optimistic about the ability to use prompts to get us fairly far with this. More powerful/agentic systems will help a lot to actually execute those prompts at scale, but the core technical challenge seems like it could be fairly straightforward. I’ve been experimenting with LLMs to try to detect what information that they could come up with that would later surprise them. I think this is fairly measurable.
Even comment-writing is to a large extent an original seeing task, when all the relevant context would otherwise seem to be more straightforward to assemble than when writing the post itself unprompted. A good comment to a post is not a review of the whole post. It finds some point in it, looks at it in a particular way, and finds that there is a relevant observation to be made about it.
Crucially, a comment won’t be made at all if such a relevant observation wasn’t found, or else you get slop even when the commenter is human (“Great post!”). Chatbots did already achieve parity with a significant fraction of reddit-level comments (but not posts) I think, which are also not worth reading.
There’s not really anything wrong with ChatGPT’s attempt here, but it happens to have picked the same topic as a recent Numberphile video, and I think it’s instructive to compare how they present the same topic: https://www.numberphile.com/videos/a-1-58-dimensional-object
I think this is correct. Blogging isn’t the easy end of the spectrum, it actually involves solving novel problems of finding new useful viewpoints. But this answer it leaves out answering the core question: what advances will allow LLMs to produce original seeing?
If you think about how humans typically produce original seeing, I think there are relatively straightforward ways that an LLM-based cognitive architecture that can direct its own investigations, “think” about what it’d found, and remember (using continuous learning of some sort) what it’s found can do the same thing.
Or of course sometimes they just pop out a perspective that counts as original seeing relative to most readers. Then the challenge is identifying whether it is really of interest to enough of the target audience, which usually involves some elaborate thinking including chains of thought, goal-directed agency (and “executive function” or metacognition of noticing whether the current CoT is on-task and redirecting it if not, then aggregating all of the on-task CoT’s to draw conclusions...
It’s not clear when this will happen, because there isn’t much economic payoff for improving original seeing.
This is very much the same process as solving novel problems. LLMs rarely if ever do this. But there’s a clearer economic payoff: people want agents that do useful work without constant human intervention by solving minor novel problems of how to get past dead-ends in their current approaches.
So I’d predict we get human-level blogging in 1-2 years (that’s human-level, which is, on average, terrible—it just has some notable standouts).
For more detail, see almost all of my other posts. :)
My view on this is that writing a worthwhile blog post is not only a writing task, but also an original seeing task. You first have to go and find something out in the world and learn about it before you can write about it. So the obstacle is not necessarily reasoning (“look at this weird rock I found” doesn’t involve much reasoning, but could make a good blog post), but a lack of things to say.
There’s not any principled reason why an AI system, even a LLM in particular, couldn’t do this. There is plenty going on in the world to go and find out, even if you’re stuck in the internet. (And even without an internet connection, you can try and explore the world of math.) But it seems like currently the bottleneck is that LLM’s don’t have anything to say.
Maybe novels might require less of this than blog posts, but I’d guess that writing a good novel is also a task that requires a lot of original seeing.
I feel like I’ve heard this before, and can sympathize, but I’m skeptical.
I feel like this prescribes an almost magical thinking to how many blog posts are produced. The phrase “original seeing” sounds much more profound than I’m comfortable with for such a discussion.
Let’s go through some examples:
Lots of Zvi’s posts are summaries of content, done in a ways that’s fairly formulaic.
A lot of Scott Alexander’s posts read to me like, “Here’s an interesting area that blog readers like but haven’t investigated much. I read a few things about it, and have some takes that make a lot of sense upon some level of reflection.”
A lot of my own posts seem like things that wouldn’t be too hard to come up with some search process to create.
Broadly, I think that “coming up with bold new ideas” gets too much attention, and more basic things like “doing lengthy research” or “explaining to people the next incremental set of information that they would be comfortable with, in a way that’s very well expressed” gets too little.
I expect that future AI systems will get good at going from a long list of [hypotheses of what might make for interesting topics] and [some great areas, where a bit of research provides surprising insights] and similar. We don’t really have this yet, but it seems doable to me.
(I similarly didn’t agree with the related post)
We probably don’t disagree that much. What “original seeing” means is just going and investigating things you’re interested in. So doing lengthy research is actually a much more central example of this than coming up with a bold new idea is.
As I say above: “There’s not any principled reason why an AI system, even a LLM in particular, couldn’t do this.”
Thanks for the clarification!
I think some of it is that I find the term “original seeing” to be off-putting. I’m not sure if I got the point of the corresponding blog post.
In general, going forward, I’d recommend people try to be very precise on what they mean here. I’m suspicious that “original seeing” will mean different things to different people. I’d expect that trying to more precisely clarify what tasks or skills involved would make it easier to pinpoint which parts of it are good/bad for LLMs.
doing lengthy research and summarizing it is important work but not typically what I associate with “blogging”. But I think pulling that together into an attractive product uses much the same cognitive skills as producing original seeing. The missing step in the process you describe is figuring out when the research did produce surprising insights, which might be a class of novel problems (unless a general formulaic approach works and someone scaffolds that in). To the extent it doesn’t require solving novel problems, I think it’s predictably easier than quality blogging that doesn’t rely on research for the novel insights.
“The missing step in the process you describe is figuring out when the research did produce surprising insights, which might be a class of novel problems (unless a general formulaic approach works and someone scaffolds that in).”
-> I feel optimistic about the ability to use prompts to get us fairly far with this. More powerful/agentic systems will help a lot to actually execute those prompts at scale, but the core technical challenge seems like it could be fairly straightforward. I’ve been experimenting with LLMs to try to detect what information that they could come up with that would later surprise them. I think this is fairly measurable.
Even comment-writing is to a large extent an original seeing task, when all the relevant context would otherwise seem to be more straightforward to assemble than when writing the post itself unprompted. A good comment to a post is not a review of the whole post. It finds some point in it, looks at it in a particular way, and finds that there is a relevant observation to be made about it.
Crucially, a comment won’t be made at all if such a relevant observation wasn’t found, or else you get slop even when the commenter is human (“Great post!”). Chatbots did already achieve parity with a significant fraction of reddit-level comments (but not posts) I think, which are also not worth reading.
Some experimental data: https://chatgpt.com/share/67ce164f-a7cc-8005-8ae1-98d92610f658
There’s not really anything wrong with ChatGPT’s attempt here, but it happens to have picked the same topic as a recent Numberphile video, and I think it’s instructive to compare how they present the same topic: https://www.numberphile.com/videos/a-1-58-dimensional-object
I think this is correct. Blogging isn’t the easy end of the spectrum, it actually involves solving novel problems of finding new useful viewpoints. But this answer it leaves out answering the core question: what advances will allow LLMs to produce original seeing?
If you think about how humans typically produce original seeing, I think there are relatively straightforward ways that an LLM-based cognitive architecture that can direct its own investigations, “think” about what it’d found, and remember (using continuous learning of some sort) what it’s found can do the same thing.
Or of course sometimes they just pop out a perspective that counts as original seeing relative to most readers. Then the challenge is identifying whether it is really of interest to enough of the target audience, which usually involves some elaborate thinking including chains of thought, goal-directed agency (and “executive function” or metacognition of noticing whether the current CoT is on-task and redirecting it if not, then aggregating all of the on-task CoT’s to draw conclusions...
It’s not clear when this will happen, because there isn’t much economic payoff for improving original seeing.
This is very much the same process as solving novel problems. LLMs rarely if ever do this. But there’s a clearer economic payoff: people want agents that do useful work without constant human intervention by solving minor novel problems of how to get past dead-ends in their current approaches.
So I’d predict we get human-level blogging in 1-2 years (that’s human-level, which is, on average, terrible—it just has some notable standouts).
For more detail, see almost all of my other posts. :)