Owain_Evans(Owain Evans)

Up­date on Ought’s ex­per­i­ments on fac­tored eval­u­a­tion of arguments

• Owain_Evans 12 Nov 2019 13:48 UTC

  12 points

  in reply to: Vaniver's comment on: Neu­ral nets as a model for how hu­mans make and un­der­stand vi­sual art

  There’s also lots of artis­tic con­cepts where the de­pen­dence on the medium is highly significant

  Great ex­am­ples. I agree the phys­i­cal medium is re­ally im­por­tant in hu­man art: see my Sec­tion 1.3.1.

  It seems like it’s not a sur­prise that NNs would be good at per­spec­tive com­pared to hu­mans, since there’s a cleaner in­verse be­tween the per­cep­tive and the cre­ation of per­spec­tive from the GAN’s point of view than the hu­man’s (who has to use their hands to make it, rather than their in­verted eyes).

  I like the point about hands vs. “in­verted eyes”. At the same time, the GANs are trained on a huge num­ber of pho­tos, and these pho­tos ex­hibit a perfect pro­jec­tion of a 3D scene onto a finite-size 2D ar­ray. The GAN’s goal is to match these pho­tos, not to match 3D scenes (which it doesn’t know any­thing about). Hu­mans in­vented per­spec­tive be­fore hav­ing pho­tos to work with. (They did have mir­rors and prim­i­tive pro­jec­tion tech­niques.)

  I think most hu­mans have pretty good fa­cil­ity with cre­at­ing and un­der­stand­ing ‘stick figures’ that comes from train­ing on a his­tory of com­mu­ni­cat­ing with other hu­mans us­ing stick figures, rather than sim­ply gen­er­al­iz­ing from vi­sual image recog­ni­tion,

  I agree that our fa­cil­ity with stick figures prob­a­bly de­pends partly on the his­tory of us­ing stick figures. How­ever, I think our gen­eral vi­sual recog­ni­tion abil­ities make us very flex­ible. For ex­am­ple, peo­ple can quickly mas­ter new styles of ab­stract de­pic­tion that differ from the XKCD style (say in a comic or a set of art­works). Deep­Mind has a cool re­cent pa­per where they learn ab­stract styles of de­pic­tion with no hu­man imi­ta­tion or la­bel­ing.

  We might want to look for find con­cepts that are eas­ier for hu­mans than NNs; when I talk to peo­ple about ML-pro­duced mu­sic, they of­ten sug­gest that it’s hard to cap­ture the sort of de­pen­den­cies that make for good mu­sic us­ing cur­rent mod­els (in the same way that cur­rent mod­els have trou­ble mak­ing ‘good art’ that’s more than style trans­fer or re­al­is­tic faces or so on; it’s un­likely that we could hook a NN up to a De­vian­tArt ac­count and ac­cept com­mis­sions and make money).

  Cur­rently hu­mans play a ma­jor role in the in­ter­est­ing ex­am­ples of neu­ral art. Get­ting more artist-like au­ton­omy is prob­a­bly AI-com­plete, but I can imag­ine neu­ral nets be­ing more and more widely used in both vi­sual art and mu­sic. I agree there’s great po­ten­tial in neu­ral mu­sic! (I sug­gest some ex­per­i­ments in my con­clu­sion but there’s tons more that could be tried).

• Owain_Evans 11 Nov 2019 19:07 UTC

  3 points

  in reply to: Charlie Steiner's comment on: Neu­ral nets as a model for how hu­mans make and un­der­stand vi­sual art

  You’d need a third and sep­a­rate scheme to make Kand­in­skys, and then I’d just bring up an­other artist not cov­ered yet.

  Again, repli­cat­ing all hu­man art is prob­a­bly AGI-com­plete. How­ever, there are some promis­ing strate­gies for gen­er­at­ing non-rep­re­sen­ta­tional art and I’d guess artists were (im­plic­itly) us­ing some of them. Here are some pos­si­ble Sen­sory Op­ti­miza­tion ob­jec­tives:

  1. Op­ti­mize the image to be a su­per­stim­u­lus for ran­dom sets of fea­tures in ear­lier lay­ers (this was already dis­cussed).

  2. Use Style Trans­fer to con­strain the low-level fea­tures in some way. This could aim at grid-like images (Mon­drian, Kelly, Albers) or a limited set of tex­tures (Richter). This is men­tioned in Sec­tion 1.3.1.

  3. If you want the image to evoke ob­jects (with­out ex­plic­itly de­pict­ing them), then you could com­bine (1) and (2) with op­ti­miz­ing for some ob­ject la­bels (e.g. river, stairs, pole). This is sim­pler than your Kand­in­sky ex­am­ple but could still be effec­tive.

  4. In ad­di­tion to (1) and (2), op­ti­mize the image for hu­man emo­tion la­bels (hav­ing trained on a dataset with emo­tion la­bels for pho­tos). To take a sim­plis­tic ex­am­ple: peo­ple will la­bel pho­tos with lots of green or blue (e.g. for­est or sea or blue skies) as peace­ful/​calming, and so ab­stract art based on those col­ors would be la­beled similarly. Red or muddy-gray col­ors would pro­duce a differ­ent re­sponse. This ex­tends be­yond col­ors to vi­sual tex­tures, shapes, sym­me­try vs. di­s­or­der and so on. (Com­pare this Rothko to this one).

  Maybe you could train an AI on pa­tri­otic paint­ings and then it could pro­duce pa­tri­otic paint­ings, but I think only by work­ing on the­ory of mind would an AI think to pro­duce a pa­tri­otic paint­ing with­out hav­ing seen one be­fore.

  I agree with your gen­eral point about the rele­vance of the­ory of mind. How­ever, I think Sen­sory Op­ti­miza­tion could gen­er­ate pa­tri­otic paint­ings with­out train­ing on them. Sup­pose you have a dataset that’s richer than ImageNet and in­cludes hu­man emo­tion and sen­ti­ment la­bels/​cap­tions. Some pho­tos will cause pa­tri­otic sen­ti­ments: e.g. pho­tos of pa­rades or par­ties on na­tional cel­e­bra­tions, pho­tos of a na­tional sports team win­ning, pho­tos of iconic build­ings or nat­u­ral won­ders. So to cre­ate pa­tri­otic paint­ings, you would op­ti­mize for la­bels re­lat­ing to pa­tri­o­tism. If there are emo­tional in­ten­sity rat­ings for pho­tos, and pa­tri­otic scenes cause high in­ten­sity, then maybe you could get pa­tri­otic paint­ings by just op­ti­miz­ing for in­ten­sity. (Face­book has trained mod­els on a huge image dataset with In­sta­gram hash­tags—some of which re­late to pa­tri­otic sen­ti­ment. Some­one could run a ver­sion of this ex­per­i­ment to­day. How­ever, I think it’s a more in­ter­est­ing ex­per­i­ment if the pho­tos are more like ev­ery­day hu­man vi­sual per­cep­tion than care­fully crafted/​ed­ited pho­tos you’ll find on In­sta­gram.)

  I was think­ing of how some things aren’t art if they’re nor­mal sized, but if you make them re­ally big, then they’re art.

  Again, I ex­pect a richer train­ing set would con­vey lots of this in­for­ma­tion. Hu­mans would use differ­ent emo­tional/​aes­thetic la­bels on see­ing un­usu­ally large nat­u­ral ob­jects (e.g. an ab­nor­mally large dog or man, a huge tree or wa­ter­fall).

  For “limited,” I imag­ined some­thing like Den­nett’s ex­am­ple of the peo­ple on the bridge. The artist only has to paint lit­tle blobs, be­cause they know how hu­mans will in­ter­pret them.

  Some art­works de­pend on idiosyn­cratic quirks of hu­man vi­sual cog­ni­tion (e.g. op­ti­cal illu­sions). It’s prob­a­bly hard for a neu­ral net to pre­dict how hu­mans will re­spond to all such works (with­out train­ing on other images that ex­ploit the same quirk). This will limit the kind of art the Sen­sory Op­ti­miza­tion model can gen­er­ate. Still, this doesn’t un­der­mine my claim that artists are do­ing some­thing like Sen­sory Op­ti­miza­tion. For ex­am­ple, hu­mans have a bias to­wards see­ing faces in ran­dom ob­jects—parei­do­lia. By ex­ploit­ing this, artists ex­ploit an image that looks like two things at once. (The artist knows the illu­sion will work, be­cause it works on his or her own vi­sual sys­tem).

  My im­pres­sion is that Deep­Dream et al. have been trained to make vi­sual art—by hy­per­pa­ram­e­ter tun­ing (grad stu­dent de­scent).

  I think this first blog­post on Deep Dream and the origi­nal pa­per on Style Trans­fer already were already very im­pres­sive. The reg­u­lariza­tion tweak for Deep Dream is very sim­ple and quite differ­ent from what I mean by “train­ing on vi­sual art”. (It’s less sur­pris­ing that a GAN trained on vi­sual art can gen­er­ate some­thing that looks like vi­sual art—al­though it is sur­pris­ing how well they can deal with stylized images.)

• Owain_Evans 11 Nov 2019 11:31 UTC

  3 points

  in reply to: Charlie Steiner's comment on: Neu­ral nets as a model for how hu­mans make and un­der­stand vi­sual art

  I agree there’s great va­ri­ety and in­tel­lec­tual so­phis­ti­ca­tion in art. My pa­per ar­gues that the Sen­sory Op­ti­miza­tion model cap­tures *some* (not all) key prop­er­ties of vi­sual art. The model is sim­ple, easy to ex­per­i­ment with (e.g. gen­er­at­ing art-like images), and cap­tures a sur­pris­ing amount. That said, there are prob­a­bly sim­ple com­pu­ta­tional mod­els that could do bet­ter and I’d be ex­cited to see con­crete pro­pos­als.

  The pa­per does touch on some of your con­cerns. Fea­ture Vi­su­al­iza­tion can gen­er­ate non-rep­re­sen­ta­tional images (Sec­tion 1.2). I sus­pect these images could be made more aes­thetic and evoca­tive by train­ing on datasets with cap­tions that in­clude hu­man emo­tional and aes­thetic re­sponses (Sec­tion 2.3), and the same goes for art that’s strongly rooted in emo­tions (Sec­tion 2.3.3). Do you have ex­am­ples in mind when you men­tion “hu­man ex­pe­rience” and “em­bod­i­ment” and “limited agents”? I don’t re­ally ad­dress art where the artist has differ­ent knowl­edge/​un­der­stand­ing than the au­di­ence and that’s an im­por­tant topic for fur­ther work (Sec­tion 2.3.4 is re­lated).

  I agree that lots of art (in­clud­ing some paint­ing) is “heav­ily lin­guis­tic, or so­cial, or re­lies on … think­ing on the part of the au­di­ence”. Hav­ing a com­pu­ta­tional model that can gen­er­ate this kind of art is plau­si­bly AGI-com­plete. Yet (as already noted) it’s likely we can do bet­ter than my cur­rent model.

  (In gen­eral, I’m op­ti­mistic about what neu­ral nets can cre­ate by Sen­sory Op­ti­miza­tion and re­lated tech­niques. Cur­rent neu­ral nets have zero ex­pe­rience of the phys­i­cal act of paint­ing or draw­ing. They have no un­der­stand­ing of how an­i­mals or hu­mans move and act in the world or of hu­man val­ues or in­ter­ests. Yet even with zero prior train­ing on vi­sual art they can make pretty im­pres­sive images by hu­man lights. I think this was sur­pris­ing to most peo­ple both in and out­side deep learn­ing. I’m cu­ri­ous whether this was sur­pris­ing to you.)

  Re­gard­ing your last para­graph, I want to make some clar­ifi­ca­tions. I don’t ex­press a view about whether Deep Dream makes art. I claim that by com­bin­ing ideas from Deep Dream and Style Trans­fer with richer datasets we could cre­ate some­thing close to a ba­sic form of hu­man vi­sual art. I don’t claim that the cre­ative pro­cess for hu­mans is like op­ti­miza­tion by gra­di­ent de­scent. In­stead, hu­mans op­ti­mize by draw­ing on their gen­eral in­tel­li­gence (e.g. hi­er­ar­chi­cal plan­ning, an­a­lyt­i­cal rea­son­ing, etc.).

Neu­ral nets as a model for how hu­mans make and un­der­stand vi­sual art

Ma­chine Learn­ing Pro­jects on IDA

Model Mis-speci­fi­ca­tion and In­verse Re­in­force­ment Learning

• Owain_Evans 25 Aug 2007 10:40 UTC

  2 points

  on: Se­man­tic Stopsigns

  Even so, you’d hope peo­ple would no­tice that on the par­tic­u­lar puz­zle of the First Cause, say­ing “God!” doesn’t help. It doesn’t make the para­dox seem any less para­dox­i­cal even if true. How could any­one not no­tice this?

  Think­ing well is difficult, even for great philoso­phers. Hind­sight bias might skew our judg­ment here.

  “About two years later, I be­came con­vinced that there is no life af­ter death, but I still be­lieved in God, be­cause the “First Cause” ar­gu­ment ap­peared to be ir­refutable. At the age of eigh­teen, how­ever, shortly be­fore I went to Cam­bridge, I read Mill’s Au­to­bi­og­ra­phy, where I found a sen­tence to the effect that his father taught him the ques­tion “Who made me?” can­not be an­swered, since it im­me­di­ately sug­gests the fur­ther ques­tion “Who made God?” This led me to aban­don the “First Cause” ar­gu­ment, and to be­come an athe­ist.”

  – Ber­trand Rus­sell, Au­to­bi­og­ra­phy of Ber­trand Rus­sell, Vol. 1, 1967.