1) Do paperclip maximizers care about paperclip mass, paperclip count, or both?
Given the origin of paperclip maximisers as a metaphor we can expect them to maximise the paperclips based off the template they were constructed with originally. It is possible that even the specification of a paperclip is unstable under recursive improvement but somewhat less likely. Postulating agents that don’t even know what a paperclip is seems less useful as a tool for constructing counterfactuals. Agents that are that flexible with respect to what their actual goal is can be used to illustrate different decision theoretic games but there is no need to recycle ‘paperclip maximiser’ for that purpose.
It is possible that even the specification of a paperclip is unstable under recursive improvement but somewhat less likely. Postulating agents that don’t even know what a paperclip is seems less useful as a tool for constructing counterfactuals.
It is, however, useful for thinking about recursive stability in general, and thinking about designing agents to have stable goal systems.
1) Do paperclip maximizers care about paperclip mass, paperclip count, or both?
Given the origin of paperclip maximisers as a metaphor we can expect them to maximise the paperclips based off the template they were constructed with originally. It is possible that even the specification of a paperclip is unstable under recursive improvement but somewhat less likely. Postulating agents that don’t even know what a paperclip is seems less useful as a tool for constructing counterfactuals. Agents that are that flexible with respect to what their actual goal is can be used to illustrate different decision theoretic games but there is no need to recycle ‘paperclip maximiser’ for that purpose.
It is, however, useful for thinking about recursive stability in general, and thinking about designing agents to have stable goal systems.