My blog: https://furretnz.bearblog.dev
folkjam
Karma: 4
Very interesting posts! I’ve played around a bunch with various LLMs (including Sonnet 4.5) and AI tools for creative writing and although nothing I’ve seen as of yet has crossed the Sturgeon’s law boundary to what I would call ‘good’ it’s pretty wild to see how uncannily it can take on a particular style. However there’s an underlying predictability to what it writes, where it feels like you have it keep feeding it your own ideas or else it will just go on doing the most obvious thing. Prompting tricks can help a little, but not enough to give it enough originality to be worth caring about. Maybe getting it to inhabit a specific persona would help with that?
Have you tried playing around with smaller open-weight models? There are a bunch of RP focused models on HuggingFace which claim to produce less sloppy writing and I’ve gotten decent results from them, though they tend to fall apart and become incoherent when given complex scenes.
In general I’ve found fancy prompting and scaffolding techniques don’t really improve the model’s creativity all that much when trying to one-shot a story. Even when I discuss setting and plot details with the model and try to guide it towards something interesting, it mostly seems to latch onto 1 idea and go default mode for everything else. The most satisfying way for me is to use text completion to continuously generate the story a couple hundred tokens at a time (koboldcpp has a mode for that) and edit its output as I go along. At that point it feels like using the LLM as a writing aid rather than treating it as a co-writer though.
Most of the open-weight models I’ve tried out came from here: https://huggingface.co/TheDrummer