Values Weren’t Complex, Once.

The cen­tral ar­gu­ment of this post is that hu­man val­ues are only com­plex be­cause all the ob­vi­ous con­straints and goals are eas­ily fulfilled. The re­sult­ing post-op­ti­miza­tion world is deeply con­fus­ing, and leads to noise as the pri­mary driver of hu­man val­ues. This has wor­ry­ing im­pli­ca­tions for any kind of world-op­ti­miz­ing. (This isn’t a par­tic­u­larly new idea, but I am tak­ing it a bit farther and/​or in a differ­ent di­rec­tion than this post by Scott Alexan­der, and I think it is worth mak­ing clear, given the pre­vi­ously noted con­nec­tion to value al­ign­ment and effec­tive al­tru­ism.)

First, it seems clear that formerly sim­ple hu­man val­ues are now com­plex. “Help and pro­tect rel­a­tives, ba­bies, and friends” as a way to en­sure group fit­ness and sur­vival is mostly ac­com­plished, so we find com­plex eth­i­cal dilem­mas about the rel­a­tive val­ues of differ­ent be­hav­ior. “Don’t hurt other peo­ple” as a tool for en­sur­ing re­ciproc­ity has turned into com­pas­sion for hu­man­ity, an­i­mals, and per­haps other forms of suffer­ing. Th­ese are more com­plex than they could pos­si­bly have been ex­pressed in the an­ces­tral en­vi­ron­ment, given re­stricted re­sources. It’s worth look­ing at what changed, and how.

In the an­ces­tral en­vi­ron­ment, hu­mans had three ba­sic de­sires; they wanted food, fight­ing, and for­ni­ca­tion. Food is now rel­a­tively abun­dant, lead­ing to peo­ple’s com­plex prefer­ences about ex­actly which fla­vors they like most. Th­ese differ be­cause the base drive for food is overop­ti­miz­ing. Fight­ing was com­pe­ti­tion be­tween peo­ple for re­sources—and since we all have plenty, this turns into sta­tus-seek­ing in ways that aren’t par­tic­u­larly mean­ingful out­side of hu­man so­cial com­pe­ti­tion. The va­ri­eties of sig­nal­ling and counter-sig­nal­ling are the re­sult. And for­ni­ca­tion was origi­nally for pro­cre­ation, but we’re adap­ta­tion ex­e­cu­tion­ers, not fit­ness max­i­miz­ers, so we’ve short-cut­ted that with birth con­trol and pornog­ra­phy, lead­ing to an ex­plo­sion in seek­ing sex­ual va­ri­ety and in­di­vi­d­ual kinks.

Past the point where max­i­miz­ing the func­tion has a mean­ingful im­pact on the in­tended re­sult, we see the tails come apart. The goal seek­ing of hu­man na­ture, how­ever, needs to find some di­rec­tion to push the op­ti­miza­tion pro­cess. The im­pli­ca­tion from this is that hu­man­ity finds di­verg­ing goals be­cause they are past the point where the ba­sic de­sires run out. As Ran­dall Mun­roe points out in an XKCD Comic, this leads to in­creas­ingly com­plex and di­ver­gent prefer­ences for ever less mean­ingful re­sults. And that comic would be funny if it weren’t a huge prob­lem for al­ign­ing group de­ci­sion mak­ing and avoid­ing longer term prob­lems.

If this is cor­rect, the key take­away is that as hu­mans find ever fewer things to need, they in­evitably to find ever more things to dis­agree about. Even though we ex­pect con­ver­gent goals re­lated to dom­i­nat­ing re­sources, nar­rowly im­ply­ing that we want to in­crease the pool of re­sources to re­duce con­flict, hu­man val­ues might be di­ver­gent as the pool of such re­sources grows.