seems like maybe the naturalness of abstracting a cluster is the disagreement in ensemble of similar-or-shorter-length equivalent models? if your abstraction is natural enough, it always holds. if it’s merely approaching the natural abstraction, it’ll approximate it. current artificial neural networks are probably not strong enough to learn all the relevant natural abstractions to a given context, but they move towards them, some of the way.
is yudkowsky’s monster a claim about the shape of the most natural abstractions? perhaps what we really need is to not assume we know which abstractions have been confirmed to be acceptably natural. ie, perhaps his body of claims about this boils down to “oh maybe soft optimization is all one could possibly ask for until all matter is equally superintelligent, so that we don’t break decision theory and all get taken over by a selfish [?genememetic element?]” or something funky along those lines.
[??] I don’t have a term of art that generalizes these things properly; genes/memes/executable shape fragments in generality
seems like maybe the naturalness of abstracting a cluster is the disagreement in ensemble of similar-or-shorter-length equivalent models? if your abstraction is natural enough, it always holds. if it’s merely approaching the natural abstraction, it’ll approximate it. current artificial neural networks are probably not strong enough to learn all the relevant natural abstractions to a given context, but they move towards them, some of the way.
is yudkowsky’s monster a claim about the shape of the most natural abstractions? perhaps what we really need is to not assume we know which abstractions have been confirmed to be acceptably natural. ie, perhaps his body of claims about this boils down to “oh maybe soft optimization is all one could possibly ask for until all matter is equally superintelligent, so that we don’t break decision theory and all get taken over by a selfish [?genememetic element?]” or something funky along those lines.
[??] I don’t have a term of art that generalizes these things properly; genes/memes/executable shape fragments in generality