For me the post is somewhat hard to read in the same way that AI-assisted writing is. Like a combination of low signal to noise and a bunch of stylistic features that make it seem like you’re trying to dazzle me without understanding me, instead of speaking plainly. Some examples, chosen at ~random:
Which is exactly the point. When you hook up a blind, localized evolutionary proxy to generalized intelligence, the proxy does not stay literal but it unfurls, bleeding into the new ontology.
and
If biological cognition acts on its payload that violently, why model AGI as having the vastness to finally make sense of gravity while maintaining the rigidity of a bacterium seeking a glucose gradient? The engine mutates the payload. When cognition scales, goals generalize.
and
To buy the lock-in story, you need a highly contradictory creature: one reflective enough to conquer the board, but oblivious enough to never notice its terminal target is a training artifact. Godlike means, buglike ends.
and
This buys us no guarantee of human compatibility; it simply says: if there is an ultimate attractor, it’s neither human morality nor paperclips, but intelligence optimization itself.
To be clear I have sympathy to trying to write unusually/in non-plain ways (see here, and here). I think the craft of writing is important to get right, and some experimentation is good. But I also understand why many LW people don’t like it when there’s a poetic register being deployed but the metaphors don’t quite work,
For me the post is somewhat hard to read in the same way that AI-assisted writing is. Like a combination of low signal to noise and a bunch of stylistic features that make it seem like you’re trying to dazzle me without understanding me, instead of speaking plainly. Some examples, chosen at ~random:
and
and
and
To be clear I have sympathy to trying to write unusually/in non-plain ways (see here, and here). I think the craft of writing is important to get right, and some experimentation is good. But I also understand why many LW people don’t like it when there’s a poetic register being deployed but the metaphors don’t quite work,
could you be more specific? what was unclear in the passages you highlighted?