IIRC, canonical old-MIRI definition of intelligence is “intelligence is cross-domain optimization” and it captures your definition modulo emotional/easy-to-understand-for-human part?
Relentlessness comes both from “optimization as looking for the ‘best’ solution” and “cross-domaining as ignoring conventional boundaries”, resourcefulness comes from picking more resources from non-standard domains, creativity is just consequence of sufficiently long optimization in sufficiently large search space.
Yeah to be clear I don’t think I’m proposing anything new, just better articulating some of the gears inside the earlier definition.
Insofar as there (might be) a new thing here, it’s articulating a narrower mechanism of cross-domain-optimization that makes it a bit more clear what kinds of not-fully-cross-domain optimization might be safer.
(there might totally other forms of dangerously efficient crossdomain optimization that aren’t covered under “relentlessly creative resourcefulnes”, articulating counterexamples seems good)
For our purposes, “intelligence” measures an
agent’s capacity for efficient cross-domain optimization of the world according to the
agent’s preferences.
IIRC, canonical old-MIRI definition of intelligence is “intelligence is cross-domain optimization” and it captures your definition modulo emotional/easy-to-understand-for-human part?
Relentlessness comes both from “optimization as looking for the ‘best’ solution” and “cross-domaining as ignoring conventional boundaries”, resourcefulness comes from picking more resources from non-standard domains, creativity is just consequence of sufficiently long optimization in sufficiently large search space.
Yeah to be clear I don’t think I’m proposing anything new, just better articulating some of the gears inside the earlier definition.
Insofar as there (might be) a new thing here, it’s articulating a narrower mechanism of cross-domain-optimization that makes it a bit more clear what kinds of not-fully-cross-domain optimization might be safer.
(there might totally other forms of dangerously efficient crossdomain optimization that aren’t covered under “relentlessly creative resourcefulnes”, articulating counterexamples seems good)
It seems you recall correctly. The old definition seemed to additionally include “efficient”.
Muehlhauser & Salamon (2012):
Yudkowsky (2013) (also expands on this definition):
See also Efficient Cross-Domain Optimization — Eliezer Yudkowsky