Nope, not LLM output, all mine, though it’s not a great sign that it can’t be distinguished :-/ The text is a constrained writing exercise, in which I fulfill the constraint described in the collapsible section above (which left me with ≤0.5% of English vocabulary, hence the obscurity of many words). Think writing without the letter ‘e’/classic oulipo. The collapsible section contains a fairly detailed fully linked/footnoted version that discusses each word choice, and some places where I wanted to use a word but couldn’t.
LLMs can’t write this kind of text. Believe me, I’ve tried, they immediately fall on their noses and use a Greek or Persian or Hindi-originated word. I think it’s a really hard task which LLMs aren’t ready for yet.
And I’m seeing line breaks, in the block-quoted text, are you not? Ah, wait, I get it, there are no paragraph breaks, but there are ’\n’s in the text, because I thought it wasn’t worth splitting 450 words into paragraphs, and wanted to evoke a sentiment of reading a page from a novel, the dialogue is split by the newlines/per character.
Maybe all the footnotes together with every word linked to its Wiktionary entry is confusing. The collapsible section contains an annotated version of the full post, as a stand-in for spoilers because I didn’t want to have a hover spoiler because that gets annoying with long texts.
The numbered list at the end is not super important, it associates the words used to their respective language families.
I was a bit unsure whether to post this here, but given that Gwern’s Tilakkhana and October the First is Too Late were posted here too, I thought this in the same genre, about the same level of obscurity.
I was a bit unsure whether to post this here, but given that Gwern’s “Tilakkhana” and “October the First is Too Late” were posted here too, I thought this in the same genre, about the same level of obscurity.
Both of those benefited from intermingled annotations/scaffolding to keep the LLMs on track, as notes. “October” puts them in comments/collapses, and “Tilakkhana” has them in comments, following the “scansion” pseudo-code.
Given the difficulty of this constraint (where do you even get a list of PIE-valid words...?), I would not adopt a constrained-sampling approach (which used to be the standard approach to such text games but doesn’t play well with any kind of planning/inner-monologue), but rather a ‘databank’ approach, closer to how I did “trajectoid words”: write down the list of PIE-valid words and frequency-valid words, and then define a format where every word in the story has to be annotated with its ‘type’ (eg ‘c’ for ‘common’ and ‘p’ for ‘PIE’), and a permissible root word if it is not in the databank. This helps reduce the problem of valid writing to a very ‘local’ problem with a cheap self-attention check back to the databank. It also makes it easier for a reasoning model to scan over a final draft to double-check validity.
So something like ... in [c] the [c] barracks [p:bara].
You could also iteratively add in valid words to the databank to save compute; add in ‘barrack’ and ‘barracks’ and ‘barracked’ to the PIE databank and future LLM runs can just write ... in [c] the [c] barracks [p]. (If you have few enough PIE words to work with, you could ask the LLM to try to generate up front all their valid variations.) This seems important given that your footnotes indicate to me that a lot of your etymologies are too debated to expect a LLM to deliver satisfactory results to you; you are going to have to lay down by fiat what are or are not valid words/roots… In fact, given the extreme difficulty you are having in writing even a coherent sentence, you’d probably want to include a sentence databank to store all the reasonably interesting valid sentences generated. (I wouldn’t necessarily bother with paragraph or higher, given how much difficulty you’re having at the word and sentence level.) I do this a lot with poetry, like with the last poem I wrote (for Valentine’s Day), I included this:
Finally, from all poems and all drafts and all ideas, select the new lines, images, and ideas (not already in the prompt/databank) which are most beautiful, unique, interesting, or promising; I will curate them so we can draw on them in future drafts.
(Even when you don’t get any new ones you want to cheap, it’s interesting for giving you an idea how the model ‘thinks’. I tried GLM-5 the other day, and I could see from its curation that it had terrible taste, which lined up with the garbage final outputs. I’ll be sticking with Kimi K2.5 Thinking as my current outside option for now...)
After you have built up enough puzzle pieces, it should be easier for the LLM to assemble them in a bunch of ways, check the fit, and then pick the best out of 20 or 100 or whatever.
And I’m seeing line breaks, in the block-quoted text, are you not? Ah, wait, I get it, there are no paragraph breaks, but there are ’\n’s in the text, because I thought it wasn’t worth splitting 450 words into paragraphs, and wanted to evoke a sentiment of reading a page from a novel, the dialogue is split by the newlines/per character.
Mm, without paragraph breaks this looked just sorta broken and confusing. (I don’t know what you mean by “wanted to evoke a sentiment of reading a page from a novel” since novels generally have paragraph breaks)
I asked about LLMs because I wasn’t sure if your “LLMs have a really hard time writing under this constraint” quote was more like “LLMs have a hard time and I effortfully got them to do it” or “LLMs can’t, and I can” (but I wasn’t sure why the comparison was being made)
I have no objection to the exercise getting posted on LW it was just confusing
Hm, thanks for the feedback. Not sure how to change, if I bunch the sentences into paragraphs it probably becomes less readable, if I give each sentence it’s own paragraph it becomes a bit disjointed. Let me think about it.
Nope, not LLM output, all mine, though it’s not a great sign that it can’t be distinguished :-/ The text is a constrained writing exercise, in which I fulfill the constraint described in the collapsible section above (which left me with ≤0.5% of English vocabulary, hence the obscurity of many words). Think writing without the letter ‘e’/classic oulipo. The collapsible section contains a fairly detailed fully linked/footnoted version that discusses each word choice, and some places where I wanted to use a word but couldn’t.
LLMs can’t write this kind of text. Believe me, I’ve tried, they immediately fall on their noses and use a Greek or Persian or Hindi-originated word. I think it’s a really hard task which LLMs aren’t ready for yet.
And I’m seeing line breaks, in the block-quoted text, are you not? Ah, wait, I get it, there are no paragraph breaks, but there are ’\n’s in the text, because I thought it wasn’t worth splitting 450 words into paragraphs, and wanted to evoke a sentiment of reading a page from a novel, the dialogue is split by the newlines/per character.
Maybe all the footnotes together with every word linked to its Wiktionary entry is confusing. The collapsible section contains an annotated version of the full post, as a stand-in for spoilers because I didn’t want to have a hover spoiler because that gets annoying with long texts.
The numbered list at the end is not super important, it associates the words used to their respective language families.
I was a bit unsure whether to post this here, but given that Gwern’s Tilakkhana and October the First is Too Late were posted here too, I thought this in the same genre, about the same level of obscurity.
Both of those benefited from intermingled annotations/scaffolding to keep the LLMs on track, as notes. “October” puts them in comments/collapses, and “Tilakkhana” has them in comments, following the “scansion” pseudo-code.
Given the difficulty of this constraint (where do you even get a list of PIE-valid words...?), I would not adopt a constrained-sampling approach (which used to be the standard approach to such text games but doesn’t play well with any kind of planning/inner-monologue), but rather a ‘databank’ approach, closer to how I did “trajectoid words”: write down the list of PIE-valid words and frequency-valid words, and then define a format where every word in the story has to be annotated with its ‘type’ (eg ‘c’ for ‘common’ and ‘p’ for ‘PIE’), and a permissible root word if it is not in the databank. This helps reduce the problem of valid writing to a very ‘local’ problem with a cheap self-attention check back to the databank. It also makes it easier for a reasoning model to scan over a final draft to double-check validity.
So something like
... in [c] the [c] barracks [p:bara].You could also iteratively add in valid words to the databank to save compute; add in ‘barrack’ and ‘barracks’ and ‘barracked’ to the PIE databank and future LLM runs can just write
... in [c] the [c] barracks [p].(If you have few enough PIE words to work with, you could ask the LLM to try to generate up front all their valid variations.) This seems important given that your footnotes indicate to me that a lot of your etymologies are too debated to expect a LLM to deliver satisfactory results to you; you are going to have to lay down by fiat what are or are not valid words/roots… In fact, given the extreme difficulty you are having in writing even a coherent sentence, you’d probably want to include a sentence databank to store all the reasonably interesting valid sentences generated. (I wouldn’t necessarily bother with paragraph or higher, given how much difficulty you’re having at the word and sentence level.) I do this a lot with poetry, like with the last poem I wrote (for Valentine’s Day), I included this:(Even when you don’t get any new ones you want to cheap, it’s interesting for giving you an idea how the model ‘thinks’. I tried GLM-5 the other day, and I could see from its curation that it had terrible taste, which lined up with the garbage final outputs. I’ll be sticking with Kimi K2.5 Thinking as my current outside option for now...)
After you have built up enough puzzle pieces, it should be easier for the LLM to assemble them in a bunch of ways, check the fit, and then pick the best out of 20 or 100 or whatever.
Mm, without paragraph breaks this looked just sorta broken and confusing. (I don’t know what you mean by “wanted to evoke a sentiment of reading a page from a novel” since novels generally have paragraph breaks)
I asked about LLMs because I wasn’t sure if your “LLMs have a really hard time writing under this constraint” quote was more like “LLMs have a hard time and I effortfully got them to do it” or “LLMs can’t, and I can” (but I wasn’t sure why the comparison was being made)
I have no objection to the exercise getting posted on LW it was just confusing
Hm, thanks for the feedback. Not sure how to change, if I bunch the sentences into paragraphs it probably becomes less readable, if I give each sentence it’s own paragraph it becomes a bit disjointed. Let me think about it.
huh, not sure why paragraphs feel disjointed to you, feels like totally normal dialogue-heavy writing to me. made a draft with breaks here: https://www.lesswrong.com/editPost?postId=dRzsobGjtH6AyBkYp&key=a93a852f88e272373f6f8958137cc5
Fair enough, your version looks good, I’ll edit this main one to conform—done.