I’ve changed my mind about AIs developing a self-concept. I see little value in worrying about whether an AI has a self concept.
I had been imagining that an AI would avoid recursive self-improvement if it had no special interest in the source code / weights / machines that correspond to it.
I now think that makes little difference. Suppose the AI doesn’t care whether it’s improving its source code or creating a new tool starting from some other code base. As long as the result improves on its source code, we get roughly the same risks and benefits as with recursive self improvement.
I’ve changed my mind about AIs developing a self-concept. I see little value in worrying about whether an AI has a self concept.
I had been imagining that an AI would avoid recursive self-improvement if it had no special interest in the source code / weights / machines that correspond to it.
I now think that makes little difference. Suppose the AI doesn’t care whether it’s improving its source code or creating a new tool starting from some other code base. As long as the result improves on its source code, we get roughly the same risks and benefits as with recursive self improvement.