Well, the paperclip maximizer may be imperfect in some aspect.
Maybe it didn’t research cryptography, because at given time making more paperclips seemed like a better choice than researching cryptography. (All intelligent agents may at some moment face a choice between developing an abstract theory with uncertain possible future gains vs pursuing their goals more directly; and they may make a wrong choice.)
The crypto here is a bit of a red herring; you want that in adversarial contexts, but a paperclipper may not necessarily optimize much for adversaries (the universe looks very empty). However, a lot of agents are going to research error-checking and correction because you simply can’t build very advanced computing hardware without ECC somewhere in it—a good chunk of every hard drive is devoted to ECC for each sector and discs like DVD/BDs have a lot of ECC built in as well. And historically, ECC either predates the most primitive general-purpose digital computers (scribal textual checks) or closely accompanies them (eg. Shannon’s theorem), and of course we have a lot of natural examples (the redundancy in how DNA codons code for amino acids turns out to be highly optimized in an ECC sense).
So, it seems pretty probable that ECC is a convergent instrumental technique.
Well, the paperclip maximizer may be imperfect in some aspect.
Maybe it didn’t research cryptography, because at given time making more paperclips seemed like a better choice than researching cryptography. (All intelligent agents may at some moment face a choice between developing an abstract theory with uncertain possible future gains vs pursuing their goals more directly; and they may make a wrong choice.)
The crypto here is a bit of a red herring; you want that in adversarial contexts, but a paperclipper may not necessarily optimize much for adversaries (the universe looks very empty). However, a lot of agents are going to research error-checking and correction because you simply can’t build very advanced computing hardware without ECC somewhere in it—a good chunk of every hard drive is devoted to ECC for each sector and discs like DVD/BDs have a lot of ECC built in as well. And historically, ECC either predates the most primitive general-purpose digital computers (scribal textual checks) or closely accompanies them (eg. Shannon’s theorem), and of course we have a lot of natural examples (the redundancy in how DNA codons code for amino acids turns out to be highly optimized in an ECC sense).
So, it seems pretty probable that ECC is a convergent instrumental technique.
E.g. proofreading in biology