When I’m in abstraction-creation mode, I feel like I’m doing something intelligent. When yell at AI about bad notation and writing, I feel like I’m providing taste. But I often find myself giving feedback that seems incredibly basic. Things like: This is a mess, clean it up. Or: Do you need that assumption? Or: If you can’t solve this problem, why don’t you create a simpler version and try to solve it first?
On the one hand, I guess a lot of what good human managers do amounts to giving very basic feedback. On the other hand, it’s hard to imagine that I have much durable advantage in doing this kind of thing.
Well, LLMs being terrible at this harkens back all the way to the olden days of the original Cyborgism whitepaper (February 2023). I remember reading all the same takes and having all the same discussions back then: that LLMs are already basically human-level agents that can work autonomously sans needing periodic basic course-corrections from humans, which will stop being necessary aaaany day now. Yet by all indications LLMs have somehow not really gotten an iota better at this since, despite all their other achievements, in a way that strikes me as almost paradoxical.
So maybe the paradox will persist, and you have until like 2040-2050? ¯\_(ツ)_/¯
IMO, they have gotten quite a bit better, but still suffer from a lack of continual learning that would allow them to course-correct better over long tasks. It’s hard to gain new competencies when the experience never flows back into the weights after initial training.
Partly what is going on here is that the LLM does not have full access to the context to solve any particular task. The full context lives in the minds of humans, or the collective minds of many humans, or inside some institution.
When you are like ‘clean this up’. I always wonder how much of that is “clean it up so it is in line with some context that actually lives in my mind but I forgot to tell you”
It’s for this reason I don’t expect humans to be out of a job, even when ASI hits (until the entire economy is automated). There’s too much context inside of human minds that constantly needs to be elicited from you by the LLM through interacting with it.
When you are like ‘clean this up’. I always wonder how much of that is “clean it up so it is in line with some context that actually lives in my mind but I forgot to tell you”
Yeah, there’s definitely a lot of that. If we had decent BCIs, that would plausibly indeed make strong cyborgism/centaurs work. Like, I think that may actually 10x even my cursed work.
That said, humans don’t have telepathy, yet human-to-human collaboration still works better than human-to-LLM. So I don’t think it’s the whole story, LLMs are genuinely dropping the ball in some ways that humans with just as much context (e. g., a remote-worker subcontractor, communicated-with via pretty sparse, mostly text-only, channels) wouldn’t.
I suspect a lot of context lives in the collective minds of humans that is not easily or obviously inferred in a one-to-one sense from all the training data. It’s not a one-to-one mapping or many-to-one from training to outcome, there’s many ways to interpret the training data that the AI is seeing during training and also tacit knowledge/wisdom.
The latter is how I imagine you can have superintelligence but it will still fail to build an EUV machine one-shot because so much context lives in the entire supply chain and it’s nowhere written down and it has to go and each and every one of those people and observe very closely what they do, and then try it out iterate and fail.
Well, LLMs being terrible at this harkens back all the way to the olden days of the original Cyborgism whitepaper (February 2023). I remember reading all the same takes and having all the same discussions back then: that LLMs are already basically human-level agents that can work autonomously sans needing periodic basic course-corrections from humans, which will stop being necessary aaaany day now. Yet by all indications LLMs have somehow not really gotten an iota better at this since, despite all their other achievements, in a way that strikes me as almost paradoxical.
So maybe the paradox will persist, and you have until like 2040-2050? ¯\_(ツ)_/¯
IMO, they have gotten quite a bit better, but still suffer from a lack of continual learning that would allow them to course-correct better over long tasks. It’s hard to gain new competencies when the experience never flows back into the weights after initial training.
Partly what is going on here is that the LLM does not have full access to the context to solve any particular task. The full context lives in the minds of humans, or the collective minds of many humans, or inside some institution.
When you are like ‘clean this up’. I always wonder how much of that is “clean it up so it is in line with some context that actually lives in my mind but I forgot to tell you”
It’s for this reason I don’t expect humans to be out of a job, even when ASI hits (until the entire economy is automated). There’s too much context inside of human minds that constantly needs to be elicited from you by the LLM through interacting with it.
Yeah, there’s definitely a lot of that. If we had decent BCIs, that would plausibly indeed make strong cyborgism/centaurs work. Like, I think that may actually 10x even my cursed work.
That said, humans don’t have telepathy, yet human-to-human collaboration still works better than human-to-LLM. So I don’t think it’s the whole story, LLMs are genuinely dropping the ball in some ways that humans with just as much context (e. g., a remote-worker subcontractor, communicated-with via pretty sparse, mostly text-only, channels) wouldn’t.
I suspect a lot of context lives in the collective minds of humans that is not easily or obviously inferred in a one-to-one sense from all the training data. It’s not a one-to-one mapping or many-to-one from training to outcome, there’s many ways to interpret the training data that the AI is seeing during training and also tacit knowledge/wisdom.
The latter is how I imagine you can have superintelligence but it will still fail to build an EUV machine one-shot because so much context lives in the entire supply chain and it’s nowhere written down and it has to go and each and every one of those people and observe very closely what they do, and then try it out iterate and fail.