The primary foreseeable difficulty Yudkowsky offered for the value identification problem is that human value is complex.[5]
That was always a poorly posed claim. The issue is whether value is unusually or uniquely complex. An ordinary non-moral sentence like “fill a bucket” still needs additional information to be interpreted. Most lesswrongians have spent years behaving as though it was a fact that moral assertions have some extra complexity, although it was never proven (and it depends on dubious assumptions about GOFAI, incorrigibility, Foom, etc).
That was always a poorly posed claim. The issue is whether value is unusually or uniquely complex. An ordinary non-moral sentence like “fill a bucket” still needs additional information to be interpreted. Most lesswrongians have spent years behaving as though it was a fact that moral assertions have some extra complexity, although it was never proven (and it depends on dubious assumptions about GOFAI, incorrigibility, Foom, etc).