This is a very strong endorsement but I’m finding it hard to separate the general picture from RFLO:
mesa-optimization occurs when a base optimizer...finds a model that is itself an optimizer,
where
a system is an optimizer if it is internally searching through a search space (consisting of possible outputs, policies, plans, strategies, or similar) looking for those elements that score high according to some objective function that is explicitly represented within the system.
i.e. a mesa-optimizer is a learned model that ‘performs inference’ (i.e. evaluates inputs) by internally searching and choosing an output based on some objective function.
Apparently a “direct optimizer” is something that “perform[s] inference by directly choosing actions[1] to optimise some objective function”. This sounds almost exactly like a mesa-optimizer?
This is a very strong endorsement but I’m finding it hard to separate the general picture from RFLO:
where
i.e. a mesa-optimizer is a learned model that ‘performs inference’ (i.e. evaluates inputs) by internally searching and choosing an output based on some objective function.
Apparently a “direct optimizer” is something that “perform[s] inference by directly choosing actions[1] to optimise some objective function”. This sounds almost exactly like a mesa-optimizer?