Prediction: Superpersuasion in the relevant sense¹ is possible, and people who claim it isn’t will look silly in retrospect if we are around to test it.
Human brains have insane data throughput, large number of degrees of freedom, no optimization against superintelligent adversaries. If we can’t solve adversarial examples for neural nets then evolution definitely didn’t solve them for humans.
Reasons this might not work is heterogenous brain structure between humans, too much noise. But humans have some shared structure (how to quantify this is tricky, according to Claude even a bunch of cortical fine structure is shared), and there’s still enough degrees of freedom here for vast optimization. Noise is harder to defeat but if an AI gets decent feedback loop speed then it may not matter as much. Still, noise is plausibly the hardest bottleneck to overcome.
I don’t think conscious processing bottleneck is a problem, because most action-relevant stuff doesn’t actually go through the central part of the global workspace.
Also worth distinguishing data throughput (video>audio≫text), speed of feedback (again same modality ranking).
Relevant thought experiment: Could we create an infohazardous image or audioclip by doing gradient descent on a latent space with direct biofeedback? MKUltra is relevant but they were in the 50s to 70s so their science & tech was also bad.
¹: Namely, an advanced AI can convince the leadership of the company that develops it to perform ~any action, irrespectively of their beliefs. Predicated on some sort of multimodal I/O OR very long continuous interactions between the AI and the lab personnel (with some memory bank).
I agree that simple versions of superpersuasion are untenable. I recently put some serious thought into what an actual attempt at superpersuasion by a sufficiently capable agent would look like, reasoning that history is already replete with examples of successful “superpersuasion” at scale (all of the -isms).
My general conclusion is that “memetic takeover” has to be multi-layered, with different “messages” depending on the sophistication of the target, rather than a simple “Snowcrash” style meme.
I agree that simple versions of superpersuasion are untenable.
I argued the opposite (unless by “simple versions” you mean [text-only, low throughput, low feedback, low-/no-memory, high latency, low context interactions]). I’ll put your post on my [[To Read]] list.
Prediction: Superpersuasion in the relevant sense¹ is possible, and people who claim it isn’t will look silly in retrospect if we are around to test it.
Human brains have insane data throughput, large number of degrees of freedom, no optimization against superintelligent adversaries. If we can’t solve adversarial examples for neural nets then evolution definitely didn’t solve them for humans.
Reasons this might not work is heterogenous brain structure between humans, too much noise. But humans have some shared structure (how to quantify this is tricky, according to Claude even a bunch of cortical fine structure is shared), and there’s still enough degrees of freedom here for vast optimization. Noise is harder to defeat but if an AI gets decent feedback loop speed then it may not matter as much. Still, noise is plausibly the hardest bottleneck to overcome.
I don’t think conscious processing bottleneck is a problem, because most action-relevant stuff doesn’t actually go through the central part of the global workspace.
Also worth distinguishing data throughput (video>audio≫text), speed of feedback (again same modality ranking).
Relevant thought experiment: Could we create an infohazardous image or audioclip by doing gradient descent on a latent space with direct biofeedback? MKUltra is relevant but they were in the 50s to 70s so their science & tech was also bad.
Think control theory perspective, you can’t predict a game of pinball.
¹: Namely, an advanced AI can convince the leadership of the company that develops it to perform ~any action, irrespectively of their beliefs. Predicated on some sort of multimodal I/O OR very long continuous interactions between the AI and the lab personnel (with some memory bank).
I agree that simple versions of superpersuasion are untenable. I recently put some serious thought into what an actual attempt at superpersuasion by a sufficiently capable agent would look like, reasoning that history is already replete with examples of successful “superpersuasion” at scale (all of the -isms).
My general conclusion is that “memetic takeover” has to be multi-layered, with different “messages” depending on the sophistication of the target, rather than a simple “Snowcrash” style meme.
I’ve worked out a potential scenario in detail.
I argued the opposite (unless by “simple versions” you mean [text-only, low throughput, low feedback, low-/no-memory, high latency, low context interactions]). I’ll put your post on my [[To Read]] list.