Pattern

• Pattern 15 Aug 2019 20:09 UTC

  2 points

  on: A Primer on Ma­trix Calcu­lus, Part 2: Ja­co­bi­ans and other fun

  The fi­nal set of images looks a bit like some­one zoom­ing in on a map*. (The blue part of the first image looks like the head of a cat.)

  *ETA: speci­fi­cally the yel­low re­gion. (Not be­cause it’s small.)

• Pattern 15 Aug 2019 19:52 UTC

  1 point

  on: Par­tial sum­mary of de­bate with Ben­quo and Jes­si­cata [pt 1]

  Peo­ple are right/​just to fear you de­vel­op­ing poli­ti­cal lan­guage if you ap­pear to be actively

  The end of this sen­tence ap­pears to be miss­ing.

  
  

  More gen­er­ally, I ap­pre­ci­ate this post, and I think it’s a good dis­til­la­tion—as some­one who can’t read what it’s a dis­til­la­tion of.

  I also think that eval­u­at­ing dis­til­la­tion qual­ity well is eas­ier with ac­cess to the con­ver­sa­tion/​data be­ing dis­til­led.

  Ab­sent any ex­am­ples of con­ver­sa­tions be­com­ing pub­lic, it looks like dis­til­la­tion is the way things are go­ing. While I don’t have any rea­son to sus­pect there are one or more con­spir­a­cies, given this:

  It’s *also* the case that pri­vate chan­nels en­able collusion

  was brought up, I am cu­ri­ous how ro­bust dis­til­la­tions are (in­tended to be) against such things, as well as how one goes about in­cen­tiviz­ing “pub­lish­ing”. For ex­am­ple, I have a model where pre-reg­istered re­sults are bet­ter* be­cause they limit cer­tain things like pub­li­ca­tion bias. I don’t have such a model for “con­ver­sa­tions”, which, while valuable, are a differ­ent re­search paradigm. (I don’t have as much of a model for, in gen­eral, how to figure out the best thing to do, ab­sent ex­per­i­ments.)

  *”Bet­ter” in terms of re­sult strength, and not nec­es­sar­ily the best thing (in a util­i­tar­ian sense).

• Pattern 14 Aug 2019 21:41 UTC

  1 point

  on: Di­ana Fleischman and Ge­offrey Miller—Au­di­ence Q&A

  For those who pre­fer to read the post on Pu­tanu­monit:

  https://​​putanu­monit.com/​​2019/​​08/​​04/​​di­ana-fleischman-and-ge­offrey-mil­ler-au­di­ence-qa/​​

  Gen­eral:

  The post says there are 13* com­ments, but only 6* can be seen.

  *After this is posted: 14 and 7, re­spec­tively.

• Pattern 14 Aug 2019 20:43 UTC

  1 point

  in reply to: Charlie Steiner's comment on: “De­sign­ing agent in­cen­tives to avoid re­ward tam­per­ing”, DeepMind

  I feel like this same set of prob­lems gets re-solved a lot. I’m wor­ried that it’s a sign of ill health for the field.

  Maybe the prob­lem is get­ting ev­ery­one on the same page.

• Pattern 14 Aug 2019 19:07 UTC

  5 points

  in reply to: William_Darwin's comment on: William_Dar­win’s Shortform

  (This is meant to be purely illus­tra­tive, not taken se­ri­ously. Also, given how hard it was to come up with frames, it might be bet­ter to re­place us­ing lenses this way with ‘ques­tions that are always good to ask’.)

  Idea [1]____1___________2__________3

  Lenses:__Eco­nomic__Nar­ra­tive [3] Em­piri­cism [4]

  
  

  [1] Movies these days seem to be lack­ing re­al­ism[2]/​X.

  1)

  Is there not an au­di­ence for re­al­ism/​X? (Or is this a mar­ket failure?)

  Are most movies pro­duced by stu­dios that aren’t good at writ­ing re­al­ism/​X?

  Is it more ex­pen­sive to pro­duce movies which are more re­al­is­tic/​X?

  Harder to make money off of?

  Is this the re­sult of gov­ern­ment reg­u­la­tion? Self-reg­u­la­tion?

  Do the peo­ple in­volved in mak­ing movies (scriptwrit­ers, di­rec­tors, etc.) pre­fer less re­al­is­tic**/​X movies? Find it eas­ier to make such movies?

  [2] What is the al­ter­na­tive to re­al­ism that is more com­mon?

  (Real­is­ti­cally, the best way to make progress on a ques­tion like this is prob­a­bly by un­pack­ing ‘What you mean by “re­al­ism”/​X.’)

  [3] A strug­gle be­tween the forces of good and evil.

  2)

  Who con­trols Hol­ly­wood, good or evil? Both? Nei­ther?

  What rea­sons might they have for do­ing this?

  Good:

  It’s eas­ier to have the good guys win in movies if you’re less re­al­is­tic. It also de­liv­ers a par­tic­u­lar mes­sage ’you will win if you’re good, no mat­ter how ridicu­lous that sounds.

  Bad:

  It lulls peo­ple into a false sense of se­cu­rity. “All evil needs to pre­vail is ev­ery good per­son do­ing noth­ing.” As it is hard to get peo­ple to do noth­ing, the noth­ing must be ob­scured by an illu­sion of do­ing some­thing—thus, mean­ingless vi­sual me­dia, Net­flix, etc.

  Nei­ther:

  a)

  Life isn’t perfect. Peo­ple go to the movies to get away from it all/​see the peo­ple they agree with win. It doesn’t have to make sense, it just has to be en­ter­tain­ing and end hap­pily.

  b)

  Movie mak­ers don’t care about re­al­ism. Con­flicts of Good ver­sus Evil, where the good guys always win, in movies that don’t make sense aren’t about Good ver­sus Evil. They’re just an­other op­por­tu­nity for movie mak­ers to set up their side as “Good” and the other side as “Evil”. This is why movies to­day are get­ting poli­ti­cal (to the detri­ment of their qual­ity).

  [4] Look­ing at data

  3)

  Assess the qual­ity of a sam­ple of movies, per­haps across time pe­ri­ods, per­haps highly rated/​pop­u­lar movies.

  Has the fac­tor we’re in­ter­ested in changed over time?

  (Is re­al­ism go­ing down, up, in a cy­cle, or ran­domly—say, based on re­ally pop­u­lar movies com­ing out which do or don’t have fea­tures (such as re­al­ism), and then more movies like that get­ting made, Y num­ber of Years later.)

  Have other fac­tors? Are there any re­la­tion­ships in the data?

• Pattern 14 Aug 2019 18:38 UTC

  3 points

  in reply to: William_Darwin's comment on: William_Dar­win’s Shortform

  (Up­date: it might be bet­ter to just come up with a list of ques­tions that are always good to ask/​have been re­ally use­ful in the past (in other do­mains) and use that in­stead. The chart is just 1) a row of such ques­tions 2) a row where you add a check­mark af­ter you’ve an­swered that ques­tion (about the topic you’re try­ing to un­der­stand.)

  A lens/​frame/​frame­work is a way of look­ing at things. I meant this to be a sug­ges­tion to see how a lens can be ap­plied to other do­mains by con­struct­ing a chart/​check­list as fol­lows:

  There are n columns and 2 rows, where n is the num­ber of lenses. The first column tells you what is in each row. The top el­e­ment of the first column con­tains ‘the name of your idea’. The bot­tom el­e­ment of the first column can con­tain the word “lens”. The bot­tom row (af­ter the first el­e­ment) con­tains the name of each lens (that you have given it). The top row (af­ter the first el­e­ment) con­tains ei­ther blank spaces or check marks*.

  Com­ing up with a pro­ce­dure to see if you’ve thought through all the im­pli­ca­tions of a model may also be use­ful.

  *One could also put a page num­ber in it, and write about that idea through that lens on that page.

• Pattern 13 Aug 2019 5:11 UTC

  4 points

  in reply to: William_Darwin's comment on: William_Dar­win’s Shortform

  Great post by the way, it was re­ally use­ful to see all these, to­gether as one sys­tem.

• Pattern 13 Aug 2019 5:10 UTC

  2 points

  in reply to: William_Darwin's comment on: William_Dar­win’s Shortform

  and ex­pe­riences you in­ter­act with

  Re­lat­edly, I’d sug­gest ‘Things you cre­ate or skills you have.’

  you de­velop the abil­ity to elimi­nate the chan­nels/​meth­ods which pre­sent the most nega­tive feed­back.

  The least use­ful feed­back?

  Start by de­vel­op­ing ideas from ac­tivi­ties you en­joy:

  Try more things, see if you en­joy them. (Also, some­times you can learn from things you don’t like—a story with a spe­cific form of bad sto­ry­tel­ling might teach you some­thing about the right way to tell sto­ries.)

  Struc­ture of In­for­ma­tion Flows Record ev­ery idea you have:

  .

  This sec­tion didn’t have bold parts.

  Idea Rate = Num­ber of Ideas /​ Num­ber of Opportunities

  Why wouldn’t “Idea Rate = Ideas per (Unit of Time)” ? It would seem one could in­crease the amount/​rate of ideas they have, not only by in­creases their ideas per op­por­tu­nity, but also by in­creas­ing their num­ber of op­por­tu­ni­ties.

  Con­stants: Num­ber of Op­por­tu­ni­ties = 800
Ideas/​Op­por­tu­nity = Idea Rate = 20%
Good Ideas/​Idea = Suc­cess Rate = 10%

  I didn’t un­der­stand this un­til I copied it here, and the for­mat­ting clicked, and then it all made sense.

  Arm­ing your­self with a vast knowl­edge of any par­tic­u­lar situ­a­tion or topic gives you a bet­ter chance of com­ing up with the cor­rect solu­tion to a given prob­lem be­cause as your net­work of un­der­stand­ing grows,

  (Em­pha­sis added.)

  Prob­lems are im­por­tant, as is com­ing up with them/​hav­ing good sources for them.

  Each in­di­vi­d­ual you come in con­tact with is an op­por­tu­nity to glean unique and valuable in­for­ma­tion from.

  It might be use­ful to come up with frames and give them names and put them in a list, so you can do this:

  [New Idea]: (check)

  Frames [Frame 1] [Frame 2] etc.

  Re­duce Cog­ni­tive De­lay[**]

  Also see how you can im­ple­ment an idea, par­tic­u­larly—quickly. (This one can be hard.)

  More gen­er­ally, re­lease “De­lays” pe­riod. Hav­ing a model of the pro­cess can help with this. (It’s pos­si­ble the low hang­ing fruit has been picked in com­mu­ni­ca­tion tech­nol­ogy on the raw speed front*, but it’s use­ful to note how this can speed up things we do.)

  *To such an ex­tent some may find it detri­men­tal. One could com­pare the qual­ity of com­ments on twit­ter with the qual­ity of let­ters, or the qual­ity of of moves in a live chess game ver­sus one by post. (It’s also eas­ier to draw on pa­per.)

  Look­ing for HARSH crit­i­cism

  That’s hard to do with such a good idea.

  [**]####5. Gain Around Pos­i­tive Feed­back Loops a. Find a re­cep­tive au­di­ence:

  The sec­tion above is re­lated to that fact that “idea qual­ity” can be sub­jec­tive—com­ing up with ideas that sound great is all well and good, but re­al­ity is the fi­nal ar­biter. (Though this kind of de­pends on what you’re work­ing on.) Find­ing ways to im­ple­ment things or ideas/​test­ing things out might help. I’d also ask where these “first prin­ci­ples” come from.

  If you en­joy some­thing, you might not learn as much. Con­sider the pop­u­lar Lord of the Rings. Did you learn some­thing by read­ing/​watch­ing it? The first time? The n-th time?

  By en­gag­ing in more con­ver­sa­tion about your ideas, you de­velop a bet­ter grasp of why you re­ceive nega­tive feed­back about par­tic­u­lar top­ics.

  Eh. Can you re­ally change the world with an idea that doesn’t up­set peo­ple?

  This ties in to the idea that it is pos­si­ble for you to pro­duce a syn­the­sis of con­trar­ian idea.

  This could use some elab­o­ra­tion.

• Pattern 13 Aug 2019 4:10 UTC

  1 point

  AF

  on: Mesa-Op­ti­miz­ers and Over-op­ti­miza­tion Failure (Op­ti­miz­ing and Good­hart Effects, Clar­ify­ing Thoughts—Part 4)

  This isn’t quite em­bed­ded agency, but it re­quires the base op­ti­mizer to be “larger” than the mesa-op­ti­mizer, only al­low­ing mesa-sub­op­ti­miz­ers, which is un­likely to be guaran­teed in gen­eral.

  Size might be eas­ier to han­dle if some parts of the de­sign are shared. For ex­am­ple, if the mesa-op­ti­mizer’s de­sign was the same as the agent, and the agent un­der­stood it­self, and knew the mesa-op­ti­mizer’s de­sign, then it seems like them be­ing the same size wouldn’t be (as much of) an is­sue.

  Prin­ci­pal op­ti­miza­tion failures oc­cur ei­ther if the mesa-op­ti­miser it­self falls prey to a Good­hart failure due to shared failures in the model, or if the mesa-op­ti­mizer model or goals are differ­ent than the prin­ci­pal’s in ways that al­low the met­rics not to al­ign with the prin­ci­pals’ goals. (Abrams cor­rectly noted in an ear­lier com­ment that this is mis­al­ign­ment. I’m not sure, but it seems this is prin­ci­pally a ter­minol­ogy is­sue.)

  1) It seems like there’s a differ­ence be­tween the two cases. If I write a pro­gram to take the CRT, and then we both take it, and we both get the same score (and that isn’t a perfect score), be­cause it solved them the way I solve them, that doesn’t sound like mis­al­ign­ment.

  2) Cal­ling the is­sues be­tween the agents be­cause of model differ­ences “ter­minol­ogy is­sues” could also work well—this may be a lit­tle like peo­ple talk­ing past each other.

  Lastly, there are mesa-tran­sop­ti­miz­ers, where typ­i­cal hu­man types of prin­ci­ple-agent failures can oc­cur be­cause the mesa-op­ti­mizer has differ­ent goals. The other way this oc­curs is if the mesa-op­ti­mizer has ac­cess to or builds a differ­ent model than the base-op­ti­mzer.

  Some efforts re­quire mul­ti­ple par­ties be­ing on the same page. Per­haps a self driv­ing car that drives on the wrong side of the road could be called “un­syn­chro­nized” or “out of sync”. (If we re­ally like us­ing the word “al­ign­ment” the ideal state could be called “model al­ign­ment”.)

• Pattern 11 Aug 2019 3:48 UTC

  1 point

  on: In­tran­si­tive Prefer­ences You Can’t Pump

  But ar­bi­trar­ily low prob­a­bil­ities need not ex­ist.

  I be­lieve for prac­ti­cal pur­poses, “I (or you) buy a cheap lot­tery ticket, and if it’s the win­ning ticket, then you pay me $1” is low enough.

• Pattern 11 Aug 2019 3:31 UTC

  0 points

  in reply to: Raemon's comment on: Rae­mon’s Short­form Feed

  Create a ma­chine that cre­ates light­ning strikes.

• Pattern 9 Aug 2019 18:25 UTC

  2 points

  in reply to: sirjackholland's comment on: Why do hu­mans not have built-in neu­ral i/​o chan­nels?

  The first is­sue isn’t hu­mans abus­ing the sys­tem. It’s open­ing your brain/​etc. up to at­tack by par­a­sites, to say noth­ing of dis­ease.

  And that would prob­a­bly be an is­sue way be­fore the sys­tem would be de­vel­oped enough to have a lot of, if any, up­sides from func­tion­al­ity, let alone down­sides.

• Pattern 9 Aug 2019 1:32 UTC

  1 point

  in reply to: Dagon's comment on: Why do hu­mans not have built-in neu­ral i/​o chan­nels?

  I think the is­sue isn’t trial and er­ror mak­ing one (se­curely) - it’s that it’d be ex­pen­sive, and 2 par­ties would have to have it, to use it.

• Pattern 9 Aug 2019 1:25 UTC

  3 points

  on: Why do hu­mans not have built-in neu­ral i/​o chan­nels?

  What’s the clos­est thing to this we see in any species?

  I recom­mend ask­ing this as its own ques­tion, and note that com­mu­ni­ca­tion across species may be more in­ter­est­ing.

• Pattern 7 Aug 2019 19:26 UTC

  3 points

  in reply to: romeostevensit's comment on: Help fore­cast study repli­ca­tion in this so­cial sci­ence pre­dic­tion market

  Links still work, I think it’s a cross-post­ing is­sue.

  Link.

  (De­tails: If you click on it, it goes to: https://​​www.less­wrong.com/​​posts/​​Xzetp­pcF8BNoDqFBs/​​https://​​www.less­wrong.com/​​out?url=https%3A%2F%2Fwww.repli­ca­tion­mar­kets.com%2F

  If you shorten that to:

  https://​​www.less­wrong.com/​​out?url=https%3A%2F%2Fwww.repli­ca­tion­mar­kets.com%2F

  It redi­rects to:

  https://​​www.repli­ca­tion­mar­kets.com/​​ )

• Pattern 7 Aug 2019 18:06 UTC

  1 point

  on: In defense of Or­a­cle (“Tool”) AI research

  What does it take for some­thing to qual­ify as agent AI?

  Con­sider some­thing like Siri. Sup­pose you could not only ask for in­for­ma­tion (“What will the weather be like to­day?”), but you could also ask for ac­tion (“Call 911/​the hos­pi­tal”). Does this cross the line from “Or­a­cle” to “Agent”?

• Pattern 7 Aug 2019 18:00 UTC

  3 points

  in reply to: moridinamael's comment on: In defense of Or­a­cle (“Tool”) AI research

  when the ca­pa­bil­ities of an un­con­strained Agent AI will es­sen­tially always sur­pass those of an Or­a­cle-hu­man syn­the­sis.

  Nit­pick: the ca­pa­bil­ities of ei­ther a) un­con­strained Agent AI/​s, or b) Ar­tifi­cial Agent-hu­man syn­the­sis, will es­sen­tially always sur­pass those of an Or­a­cle-hu­man syn­the­sis. We might have to work our way up to AIs with­out hu­mans be­ing more effec­tive.

• Pattern 7 Aug 2019 17:19 UTC

  1 point

  in reply to: Raemon's comment on: Rae­mon’s Short­form Feed

  Per­haps de­gree of in­vest­ment. Con­sider the amount of time it takes for some­one to grow up, and the effort in­volved in teach­ing them (how to talk, read, etc.). (And be­fore that, preg­nancy.)

  There is at least one book that plays with this—the pro­tag­o­nist finds out they were stolen from ‘their fam­ily’ as a baby (or re­ally small child), and the peo­ple who stole them raised them, and up to that point they had no idea. I don’t re­mem­ber the ti­tle.

• Pattern 7 Aug 2019 6:31 UTC

  1 point

  in reply to: Said Achmiz's comment on: Open & Wel­come Thread Au­gust 2019

  A more ex­treme ver­sion of the in­ter­per­sonal is that (one might sup­pose) you could have two (oth­er­wise*) iden­ti­cal uni­verses such that in the first Bob an­swers “3” and in the sec­ond Bob an­swers “5″, where both Bob’s feel (cur­rently) the same way (largely*), but a) used differ­ent refer­ence points within their life, or b) fo­cus on differ­ent things—per­haps in uni­verse 1 Bob thinks or says “my life could be bet­ter − 3”, but in uni­verse 2 Bob thinks or says “my life could be worse** − 5′.

  *This might re­quire some as­sump­tions about Bob, that don’t nec­es­sar­ily ap­ply to ev­ery­one.

  **Per­haps in uni­verse 2, it oc­curs to Bob that he’s never had can­cer, or any­thing of similar mag­ni­tude.

• Pattern 7 Aug 2019 3:57 UTC

  1 point

  on: Per­cent re­duc­tion of gun-re­lated deaths by color of gun.

  As­sume all guns were pink by law to­mor­row.

  All guns, or all new guns made from here on out?