I’ve found it interesting to run my writing through Google Translate: English → Japanese → English. I’ve found that the result is better in certain ways. For example:
It is more direct and uses fewer idioms.
Vague phrasing is usually eliminated.
Pronouns are often also eliminated, by positioning nouns closer to discussion about them.
The vocabulary tends to be more fitting.
It almost feels like Legos snapping into place.
This almost certainly happens because English and Japanese differ significantly in vocabulary and structure. The machine translation maps ideas to their closest equivalents, using context to choose the right word. When mapped back, this process repeats, finding a word with not only the right meaning, but the right grade of meaning and connotations. It also forces phrases to be rearranged within a sentence (or sometimes even between sentences), since English is SVO (Subject-Verb-Object word order) while Japanese is SOV (Subject-Object-Verb). Indirect meanings conveyed originally through sentence structure thus need to be made explicit. When reconstructing sentences Google Translate also tends to choose simpler phrasings.
I do like the exactness and directness of the result, but it is often a bit verbose. It’s also atrocious for creative writing. But if you just want to get your ideas across as clearly as you can, hitting “⇄” twice is very good at doing that. Of course, sometimes the translations are wrong. But Google Translate is now good enough that I’ve found the errors are almost always because my original text was itself ambiguous. If nothing else, this is great tool for testing whether others will interpret what you write in the way you meant it.
Have you experimented with using languages other than Japanese for that?
(longer cycles e.g. English→Mandarin→Arabic→English might also be interesting to try, in principle, though I’d guess that in the current incarnation of Google Translate Mandarin→Arabic is just Mandarin→English→Arabic in a trenchcoat anyway)
I recently came across Backpack Language Models and wanted to share it in case any AI interpretability people have not seen it. (I have yet to see this posted on LessWrong.)
The main difference between a backpack model and an LLM is that it enforces a much stricter rule to map inputs’ embeddings to output logits. Most LLMs allow the output logits to be an arbitrary function of the inputs’ embeddings; a backpack model requires the output logits to be a linear transformation of a linear combination of the input embeddings. The weights for this linear combination are parameterized by a transformer.
The nice thing about backpack models is that they are somewhat easier to interpret/edit/control: The output logits are a linear combination of the inputs’ embeddings, so you can directly observe how changing the embeddings changes the outputs.
I’ve found it interesting to run my writing through Google Translate: English → Japanese → English. I’ve found that the result is better in certain ways. For example:
It is more direct and uses fewer idioms.
Vague phrasing is usually eliminated.
Pronouns are often also eliminated, by positioning nouns closer to discussion about them.
The vocabulary tends to be more fitting.
It almost feels like Legos snapping into place.
This almost certainly happens because English and Japanese differ significantly in vocabulary and structure. The machine translation maps ideas to their closest equivalents, using context to choose the right word. When mapped back, this process repeats, finding a word with not only the right meaning, but the right grade of meaning and connotations. It also forces phrases to be rearranged within a sentence (or sometimes even between sentences), since English is SVO (Subject-Verb-Object word order) while Japanese is SOV (Subject-Object-Verb). Indirect meanings conveyed originally through sentence structure thus need to be made explicit. When reconstructing sentences Google Translate also tends to choose simpler phrasings.
I do like the exactness and directness of the result, but it is often a bit verbose. It’s also atrocious for creative writing. But if you just want to get your ideas across as clearly as you can, hitting “⇄” twice is very good at doing that. Of course, sometimes the translations are wrong. But Google Translate is now good enough that I’ve found the errors are almost always because my original text was itself ambiguous. If nothing else, this is great tool for testing whether others will interpret what you write in the way you meant it.
Do you practice what you preach? I.e. did you run this very post through the same pipeline? :)
I did, but with significant editing afterwards!
Have you experimented with using languages other than Japanese for that?
(longer cycles e.g. English→Mandarin→Arabic→English might also be interesting to try, in principle, though I’d guess that in the current incarnation of Google Translate Mandarin→Arabic is just Mandarin→English→Arabic in a trenchcoat anyway)
I recently came across Backpack Language Models and wanted to share it in case any AI interpretability people have not seen it. (I have yet to see this posted on LessWrong.)
The main difference between a backpack model and an LLM is that it enforces a much stricter rule to map inputs’ embeddings to output logits. Most LLMs allow the output logits to be an arbitrary function of the inputs’ embeddings; a backpack model requires the output logits to be a linear transformation of a linear combination of the input embeddings. The weights for this linear combination are parameterized by a transformer.
The nice thing about backpack models is that they are somewhat easier to interpret/edit/control: The output logits are a linear combination of the inputs’ embeddings, so you can directly observe how changing the embeddings changes the outputs.