>> I’ve been trying to understand and express why I find natural language alignment … so much more promising >> than any other alignment techniques I’ve found.
Could it be that we humans have millennia of experience aligning our new humans (children) using this method? Whereas every other method is entirely new to us, and has never been applied to a GI even if it has been tested on other AI systems; thus, predictions of outcomes are speculative.
But it still seems like there is something missing from specifying goals directly via expression through language or even representational manipulation. If the representations themselves do not contain any reference to motivational structure (i.e., they are “value free” representations), then the goals will not be particularly stable. Johnny knows that it’s bad to hit his friends because Mommy told him so, but he only cares because it’s Mommy who told him, and he has a rather strong psychological attachment to Mommy.,
It’s worse than that. https://arxiv.org/abs/1604.06963