I’ve been doing similar things with my day-to-day work like making stuff in CSS/Bootstrap or Excel, and my hobbies like mucking about in Twine or VCV Rack, and have noticed:
a similar vibe of there seems to be a “goldilocks prompt narrowness” that gives really good results
that goldilocks band is different for different topics
plausible-sounding errors sneak in at all levels except the broadest, where it tends more towards very hedged “fluffy” statements like “be careful!”
However, if you treat it almost like a student, and inform it of the errors/consequences of whatever it suggested, it’s often surprisingly good at correcting the error, but here is where differences between how much it “understands” domains like “CSS” vs. “Twine’s Harlowe 3.3.4 macro format” become easier to see- it seems much more likely to make up function and features of Harlowe that resemble things from more popular languages.
For whatever reason, it’s really fun to engage it on things you have expertise in and correct it and/or rubber duck off of it. It gives you a weird child of expertise and outsider art.
I’ve been doing similar things with my day-to-day work like making stuff in CSS/Bootstrap or Excel, and my hobbies like mucking about in Twine or VCV Rack, and have noticed:
a similar vibe of there seems to be a “goldilocks prompt narrowness” that gives really good results
that goldilocks band is different for different topics
plausible-sounding errors sneak in at all levels except the broadest, where it tends more towards very hedged “fluffy” statements like “be careful!”
However, if you treat it almost like a student, and inform it of the errors/consequences of whatever it suggested, it’s often surprisingly good at correcting the error, but here is where differences between how much it “understands” domains like “CSS” vs. “Twine’s Harlowe 3.3.4 macro format” become easier to see- it seems much more likely to make up function and features of Harlowe that resemble things from more popular languages.
For whatever reason, it’s really fun to engage it on things you have expertise in and correct it and/or rubber duck off of it. It gives you a weird child of expertise and outsider art.