This “imitating an optimizer” / “optimizing to imitate” dichotomy seems unnecessarily confusing to me. Isn’t it just inner alignment / inner misalignment (with the human behavior you’re being trained on)? If you’re imitating an optimizer, you’re still an optimizer.
I agree with this. If the key idea is, for example, optimising imitators generalise better than imitations of optimisers, or for a second example that they pursue simpler goals, it seems to me that it’d be better just to draw distinctions based on generalisation or goal simplicity and not on optimising imitators/imitations of optimisers.
Sorry, I should be more specific. We are talking about AGI Safety, it seems unlikely that running narrow AI faster gets you AGI. I’m not sure if you disagree with that. I don’t understand what you mean by “imitations of augmented of humans” and “planning against a human-level imitation”.
This “imitating an optimizer” / “optimizing to imitate” dichotomy seems unnecessarily confusing to me. Isn’t it just inner alignment / inner misalignment (with the human behavior you’re being trained on)? If you’re imitating an optimizer, you’re still an optimizer.
I agree with this. If the key idea is, for example, optimising imitators generalise better than imitations of optimisers, or for a second example that they pursue simpler goals, it seems to me that it’d be better just to draw distinctions based on generalisation or goal simplicity and not on optimising imitators/imitations of optimisers.
Sorry, I should be more specific. We are talking about AGI Safety, it seems unlikely that running narrow AI faster gets you AGI. I’m not sure if you disagree with that. I don’t understand what you mean by “imitations of augmented of humans” and “planning against a human-level imitation”.