What does an agent which has “only instrumental goals” mean? What is it optimizing for? It doesn’t seem to me like a proposal specific enough to consider whether an ASI optimizing for it would lead to good outcomes, let alone being able to build an ASI with that goal (which seems to be the harder part of the problem). If you have a more specific idea for this though then I’d like to see it.
What does an agent which has “only instrumental goals” mean? What is it optimizing for? It doesn’t seem to me like a proposal specific enough to consider whether an ASI optimizing for it would lead to good outcomes, let alone being able to build an ASI with that goal (which seems to be the harder part of the problem). If you have a more specific idea for this though then I’d like to see it.