I think early AGI may actually end up being about designing organizations that robustly pursue metrics that their (flawed, unstructured, chaotically evolved) subagents don’t reliably directly care about. Molochean equilibrium fixation and super-agent alignment may turn out to be the same questions.
I think early AGI may actually end up being about designing organizations that robustly pursue metrics that their (flawed, unstructured, chaotically evolved) subagents don’t reliably directly care about. Molochean equilibrium fixation and super-agent alignment may turn out to be the same questions.