Foundation Models tend to have a more limited type of orthogonality—they’re good at pursuing any goal that’s plausible under the training distribution, meaning they can pursue any goal that humans would plausibly have (with some caveats I guess). This is most true without outcome-based RL on top of the foundation model, but I’d guess some of the orthogonality transfers through RL.
Foundation Models tend to have a more limited type of orthogonality—they’re good at pursuing any goal that’s plausible under the training distribution, meaning they can pursue any goal that humans would plausibly have (with some caveats I guess). This is most true without outcome-based RL on top of the foundation model, but I’d guess some of the orthogonality transfers through RL.