Thank you for your thoughts, I think you are supplying valuable nuance. In private conversation I do see a general path by which this offers a strategy for capabilities enhancement, but I also think it’s sufficiently low-hanging fruit that I’d be surprised if a complete hobbyist like myself discovered a way to contribute much of anything to AI capabilities research. Then again, I guess interfacing between GPT-4-quality LLMs and traditional software is a new enough tool to explore that maybe there is enough low-hanging fruit for even a hobbyist to pluck. I agree with you that it would be ideal if there was a closed but constructive community to interface with on these issues, and I’m such a complete hobbyist that I wouldn’t know about such a group even if it existed, which is why I asked. I’ll give it some more thought.
Ah, I wasn’t really intending to make a strong claim about whether your specific idea is likely to be useful in pushing the capabilities frontier or not, just commenting generally about this general class of research (which again I think is plausibly net positive to do and publish on LW).
I do think you’re possibly selling yourself short / being overmodest, though. Inexperienced researchers often start out with the same bad ideas (in alignment or other fields), and I’ve seen others claim that any hobbyist or junior researcher shouldn’t worry about exfohazards because they’re not experienced enough to have original ideas yet. This is maybe true / good advice for some overconfident newbies, but isn’t true in general for everyone.
Also, this is not just true of alignment, but applies to academic research more generally: if a first year grad student has some grand new Theory of Everything that they think constitutes a paradigm shift or contradicts all previous work, they’re probably full of it and need to read some more whitepapers or textbooks. If they just have some relatively narrow technical idea that combines two things from different domains (e.g. LLMs and DAGs) in a valid way, it might constitute a genuinely novel, perhaps even groundbreaking insight. Or they might have just overlooked some prior work, or the idea might not pan out, or it might just not be very groundbreaking after all, etc. But in general, first year grad students (or hobbyists) are capable of having novel insights, especially in relatively young and fast-moving fields like AI and alignment.
Thank you for your thoughts, I think you are supplying valuable nuance. In private conversation I do see a general path by which this offers a strategy for capabilities enhancement, but I also think it’s sufficiently low-hanging fruit that I’d be surprised if a complete hobbyist like myself discovered a way to contribute much of anything to AI capabilities research. Then again, I guess interfacing between GPT-4-quality LLMs and traditional software is a new enough tool to explore that maybe there is enough low-hanging fruit for even a hobbyist to pluck. I agree with you that it would be ideal if there was a closed but constructive community to interface with on these issues, and I’m such a complete hobbyist that I wouldn’t know about such a group even if it existed, which is why I asked. I’ll give it some more thought.
Ah, I wasn’t really intending to make a strong claim about whether your specific idea is likely to be useful in pushing the capabilities frontier or not, just commenting generally about this general class of research (which again I think is plausibly net positive to do and publish on LW).
I do think you’re possibly selling yourself short / being overmodest, though. Inexperienced researchers often start out with the same bad ideas (in alignment or other fields), and I’ve seen others claim that any hobbyist or junior researcher shouldn’t worry about exfohazards because they’re not experienced enough to have original ideas yet. This is maybe true / good advice for some overconfident newbies, but isn’t true in general for everyone.
Also, this is not just true of alignment, but applies to academic research more generally: if a first year grad student has some grand new Theory of Everything that they think constitutes a paradigm shift or contradicts all previous work, they’re probably full of it and need to read some more whitepapers or textbooks. If they just have some relatively narrow technical idea that combines two things from different domains (e.g. LLMs and DAGs) in a valid way, it might constitute a genuinely novel, perhaps even groundbreaking insight. Or they might have just overlooked some prior work, or the idea might not pan out, or it might just not be very groundbreaking after all, etc. But in general, first year grad students (or hobbyists) are capable of having novel insights, especially in relatively young and fast-moving fields like AI and alignment.