It’s interesting to compare the first two points: novel math derivations and remixing old artwork can seem like disparate paths to greater understanding. Yet often, ‘novel’ math derivations are more like the artistic remixes, or pastiches. Gian-Carlo Rota, MIT math & philosophy prof, referenced two ways to come across as genius: either keep a bag of tricks and apply them to new problems, or keep a bag of problems and apply them to new tricks.
Eliezer’s discussion about his work was interesting too, I hadn’t seen that before. Rota also spoke of scientific popularization as your mentioned, saying you’re more likely to be remembered for expository work than for original contributions:
Allow me to digress with a personal reminiscence. I sometimes publish in a branch of philosophy called phenomenology. After publishing my first paper in this subject, I felt deeply hurt when, at a meeting of the Society for Phenomenology and Existential Philosophy, I was rudely told in no uncertain terms that everything I wrote in my paper was well known. This scenario occurred more than once, and I was eventually forced to reconsider my publishing standards in phenomenology.
It so happens that the fundamental treatises of phenomenology are written in thick, heavy philosophical German. Tradition demands that no examples ever be given of what one is talking about. One day I decided, not without serious misgivings, to publish a paper that was essentially an updating of some paragraphs from a book by Edmund Husserl, with a few examples added. While I was waiting for the worst at the next meeting of the Society for Phenomenology and Existential Philosophy, a prominent phenomenologist rushed towards me with a smile on his face. He was full of praise for my paper, and he strongly encouraged me to further develop the novel and original ideas presented in it.
Here’s the source from which I found Rota’s speech—and the fact that I wouldn’t have known of those ideas otherwise—validates the usefulness of repackaging good ideas again! And you’re right that choosing what to curate is a form of originality; choosing the best out of several AI text generations is you applying your taste and sense of relevance, or in other words, bits of selection pressure. So both human contribution and human selectivity can indicate originality. But the same could go for sampling past human work too, in art, math, or otherwise.
I loved this post. Its overall presentation felt like a text version of a Christopher Nolan mind-bender.
The crescendo of clues about the nature of the spoiler: misattributed/fictional quotes; severe link rot even though the essay was just freshly published; the apparently ‘overwhelming’ level of academic, sophisticated writing style and range of references; the writing getting more chaotic as it talks about itself getting more chaotic. And of course, the constant question of what sort of spoiler could possibly ‘alter’ the meaning of the entire essay.
I loved the feeling of Inception near the end of the essay when, in the analyst’s voice, it confirms the reader’s likely prediction that it was written by AI, only to reveal how that ‘analyst’ section was also written by AI. Or rather that the voice fluidly changes between AI and analyst, first- and third-person. And when you finally feel like you’re on solid ground, the integrity of the essay breaks down; “<!--TODO-->” tags make you contend with how no part is certainly all-human or all-AI, and so, does it even matter who wrote it.
Returning to the spoiler and initial paragraphs after finishing the essay, and getting a profound, contextualized appreciation for what it means. You realize that the essay achieved what it told you it set out to; to convey a salient point through apparent nonsense, validating that such nonsense can be useful, as it explains the process of generating the nonsense. Or in the essay’s words, “[the] string of text can talk about itself [as it] unmask[s] the code hidden within itself.”
The post also shared concepts I now use when thinking about language. My favourite being quantum poetry, associating the artificial (and ‘next-token prediction’) to the humanistic:
My main response to the essay’s content, is that still, a human in the loop seemed to still be the primary engine for most of the art in the essay. From my understanding of critical rationalism, personhood is mapped to the ability to creatively conjecture and criticize ideas to generate testable, hard to vary explanations of things.
This essay depended on a human analyst to evaluate and criticize (by some sense of ‘relevance’) which generation was valid enough to continue into the main branch of the essay. The essay also depended on a human to decide which original conjecture to write about (also by some sense of what’s ‘interesting’).
Therefore, it seems to me that AGI is still far from automating both of humans’ conjecture and criticism capacities. However, the holistic artistry the essay did push me to consider AGI’s validity more than other text I’ve read, and in that sense, it achieved what it meant to: to connect my prior thoughts to some new idea—both in the real domain—through ‘babble’ of the the imaginary domain.