This is an extremely refreshing take as it validates feelings I’ve been having ever since reading https://ghuntley.com/stdlib/ last week and trying to jump back into AI-assisted development. Of course I’m lacking many programming skills and experience to make the most of it, but I felt like I wasn’t actually getting anywhere. I found 3 major failure points which have made me consider dropping the project altogether:
I couldn’t find anything in Zed that would let me enable the agent to automatically write new rules for itself, and I couldn’t find if that was actually doable in cursor either (except through memories, which is paid and doesn’t seem under user control). If I have to manually enter the rules, that’s a significant hurdle in the cyborg future I was envisioning.
(more to the point) I absolutely have not even come close to bootstrapping this self-reinforcing capabilities growth I imagined. Certainly not getting any of my LLM tools to really understand (or at least use in their reasoning) the concept of evolving better agents by developing the rules/promp/stdlib together. They can repeat back my goals and guidelines but they don’t seem to use it.
As you said: they seem to often be lying just to fit inside a technically compliant response, selectively ignoring instructions where they think they can get away with it. The whole thing depends on them being rigorous and precise and (for lack of a better word) respectful of my goals, and this is not that.
I am certainly open to the idea that I’m just not great at it. But the way I see people refer to creating rules as a “skill issue” rubs me the wrong way because either: they’re wrong, and it’s an issue of circumstances or luck or whatever; or they’re wrong because the system prompt isn’t actually doing as much as they think; or they’re right, but it’s something you need top ~1% skill level in to get any value out of, which is disingenuous (like saying it’s a skill issue if you’re not climbing K2… yes it is, but that misses the point wildly).
This is an extremely refreshing take as it validates feelings I’ve been having ever since reading https://ghuntley.com/stdlib/ last week and trying to jump back into AI-assisted development. Of course I’m lacking many programming skills and experience to make the most of it, but I felt like I wasn’t actually getting anywhere. I found 3 major failure points which have made me consider dropping the project altogether:
I couldn’t find anything in Zed that would let me enable the agent to automatically write new rules for itself, and I couldn’t find if that was actually doable in cursor either (except through memories, which is paid and doesn’t seem under user control). If I have to manually enter the rules, that’s a significant hurdle in the cyborg future I was envisioning.
(more to the point) I absolutely have not even come close to bootstrapping this self-reinforcing capabilities growth I imagined. Certainly not getting any of my LLM tools to really understand (or at least use in their reasoning) the concept of evolving better agents by developing the rules/promp/stdlib together. They can repeat back my goals and guidelines but they don’t seem to use it.
As you said: they seem to often be lying just to fit inside a technically compliant response, selectively ignoring instructions where they think they can get away with it. The whole thing depends on them being rigorous and precise and (for lack of a better word) respectful of my goals, and this is not that.
I am certainly open to the idea that I’m just not great at it. But the way I see people refer to creating rules as a “skill issue” rubs me the wrong way because either: they’re wrong, and it’s an issue of circumstances or luck or whatever; or they’re wrong because the system prompt isn’t actually doing as much as they think; or they’re right, but it’s something you need top ~1% skill level in to get any value out of, which is disingenuous (like saying it’s a skill issue if you’re not climbing K2… yes it is, but that misses the point wildly).