Maybe you want to maximise paperclips too
As most LWers will know, Clippy the Paperclip Maximiser is a superintelligence who wants to tile the universe with paperclips. The LessWrong wiki entry for Paperclip Maximizer says that:
The goal of maximizing paperclips is chosen for illustrative purposes because it is very unlikely to be implemented
I think that a massively powerful star-faring entity—whether a Friendly AI, a far-future human civilisation, aliens, or whatever—might indeed end up essentially converting huge swathes of matter in to paperclips. Whether a massively powerful star-faring entity is likely to arise is, of course, a separate question. But if it does arise, it could well want to tile the universe with paperclips.
Let me explain.
To travel across the stars and achieve whatever noble goals you might have (assuming they scale up), you are going to want energy. A lot of energy. Where do you get it? Well, at interstellar scales, your only options are nuclear fusion or maybe fission.
Iron has the strongest binding energy of any nucleus. If you have elements lighter than iron, you can release energy through nuclear fusion—sticking atoms together to make bigger ones. If you have elements heavier than iron, you can release energy through nuclear fission—splitting atoms apart to make smaller ones. We can do this now for a handful of elements (mostly selected isotopes of uranium, plutonium and hydrogen) but we don’t know how to do this for most of the others—yet. But it looks thermodynamically possible. So if you are a massively powerful and massively clever galaxy-hopping agent, you can extract maximum energy for your purposes by taking up all the non-ferrous matter you can find and turning it in to iron, getting energy through fusion or fission as appropriate.
You leave behind you a cold, dark trail of iron.
That seems a little grim. If you have any aesthetic sense, you might want to make it prettier, to leave an enduring sign of values beyond mere energy acquisition. With careful engineering, it would take only a tiny, tiny amount of extra effort to leave the iron arranged in to beautiful shapes. Curves are nice. What do you call a lump of iron arranged in to an artfully-twisted shape? I think we could reasonably call it a paperclip.
Over time, the amount of space that you’ve visited and harvested for energy will increase, and the amount of space available for your noble goals—or for anyone else’s—will decrease. Gradually but steadily, you are converting the universe in to artfully-twisted pieces of iron. To an onlooker who doesn’t see or understand your noble goals, you will look a lot like you are a paperclip maximiser. In Eliezer’s terms, your desire to do so is an instrumental value, not a terminal value. But—conditional on my wild speculations about energy sources here being correct—it’s what you’ll do.