lifelonglearner

Jane Street In­te­view Guide (Notes on Prob­a­bil­ity, Mar­kets, etc.)

• lifelonglearner 14 Sep 2019 6:54 UTC

  2 points

  in reply to: Raemon's comment on: Rae­mon’s Short­form Feed

  There’s jquery UI which maybe counts?

• lifelonglearner 8 Sep 2019 4:47 UTC

  13 points

  on: Owen Another Thing

  Ben Pace has a new post up on LessWrong that’s ask­ing about good ex­er­cises for ra­tio­nal­ity /​ gen­eral LW-ad­ja­cent stuff. I think this is a good thing to put up a bounty for, and I started think­ing about what makes a good ex­er­cise. Ex­er­cises are good be­cause they help you fur­ther the de­velop the ma­te­rial; they give you op­por­tu­ni­ties to put what­ever rele­vant skill to use.

  There are differ­ing lev­els of what you can be try­ing to as­sess:

  • Iden­ti­fy­ing the cor­rect idea from a group of differ­ent ones

  • Sum­ma­riz­ing the cor­rect idea

  • Trans­fer­ring the idea to some­one else

  • Ac­tu­ally demon­strat­ing what­ever skill it is (if it’s some­thing you can do)

  • Ac­tu­ally us­ing the skill to de­duce some­thing else (if it’s a model thing)

  I think there’s a good set of stuff to dive into here about the dis­tinc­tion be­tween op­ti­miz­ing for ped­a­gogy ver­sus effec­tive­ness. In the most stark case, you want to teach peo­ple us­ing less po­tent ver­sions of some­thing, at least at first. Think not just train­ing wheels on a bike, but suc­ces­sively more ad­vanced mod­els for physics or ar­ith­metic. There’s a grad­ual shift hap­pen­ing.

  More than that, I won­der if the two an­gles are greatly or­thog­o­nal.

  Any­way, back to the origi­nal idea at hand. When you give peo­ple ex­er­cises, there’s a sense of broad vs nar­row that seems im­por­tant, but I’m still teas­ing it out. In one sense, you can think of tests that do mul­ti­ple choice vs open-ended an­swers. But it’s not like mul­ti­ple-choice ques­tions have to suck. You could give peo­ple very plau­si­ble-sound­ing an­swers which re­quire them to do a lot of work to de­ter­mine which one is cor­rect. Similarly, open-ended ques­tions could al­low for bul­lshit­ting.

  It’s not ex­actly the for­mat, but what sort of work it in­duces.

  At the very least, it’s about push­ing for more Gen­er­a­tive con­tent. But be­yond that, it gets into ped­a­gogy ques­tions:

  1. How can you give ques­tions which in­crease in difficulty?

     1. What does difficulty cor­re­spond to? If some­thing is “hard to figure out”, what is that qual­ity refer­ring to?

  2. If you give open-ended ques­tions, how can you as­sess the an­swers you get?

  3. How much of this is cov­ered already by the teach­ing liter­a­ture?

• lifelonglearner 8 Sep 2019 1:56 UTC

  4 points

  on: Ra­tion­al­ity Ex­er­cises Prize of Septem­ber 2019 ($1,000)

  I re­cently wrote about three things you can try with cards to see what your in­ter­nal cal­ibra­tion feels like. They have some ques­tion prompts, but the gist of it is some­thing to do, rather than some­thing with a di­rect an­swer.

  https://​​www.less­wrong.com/​​posts/​​Ktw9L67Nwq8o2kecP/​​cal­ibrat­ing-with-cards

• lifelonglearner 6 Sep 2019 20:42 UTC

  4 points

  in reply to: habryka's comment on: Does any­one else feel LessWrong is slow?

  I see! Thanks for the break­down for where the pain points are when it comes to perfor­mance. Really ap­pre­ci­ate the open­ness into where things could have gone bet­ter /​ what’s hap­pen­ing right now!

[Question] Does any­one else feel LessWrong is slow?

GPT-2: 6-Month Fol­low-Up

• lifelonglearner 18 Aug 2019 6:11 UTC

  4 points

  in reply to: habryka's comment on: Neu­ral Nets in Python 1

  Oh, wow! I didn’t re­al­ize that could have been trip­ping things up. Thank you for the for­mat­ting help!

• lifelonglearner 18 Aug 2019 3:04 UTC

  2 points

  on: Neu­ral Nets in Python 1

  META: The code block ed­i­tor wasn’t very friendly and ate up all of my tabs. I’m work­ing on bet­ter for­mat­ting, and this’ll prob­a­bly end up be­ing a post on my own blog later on, which will hope­fully also have things like syn­tax high­light­ing.

Neu­ral Nets in Python 1

• lifelonglearner 17 Aug 2019 15:48 UTC

  2 points

  in reply to: Matthew Barnett's comment on: A Primer on Ma­trix Calcu­lus, Part 3: The Chain Rule

  For sure! To be hon­est, I got a lit­tle lost read­ing your 3-part se­ries here, so I think I’ll re­visit it later on.

  I’m newer to deep learn­ing, so I think my goals are similar to yours (e.g. writ­ing it up so I have a bet­ter un­der­stand­ing of what’s go­ing on), but I’m still hash­ing out the more in­tro­duc­tory stuff.

  I’ll definitely link it here af­ter I finish!

• lifelonglearner 17 Aug 2019 6:15 UTC

  5 points

  on: A Primer on Ma­trix Calcu­lus, Part 3: The Chain Rule

  Thanks for writ­ing this se­ries!

  I’m work­ing on my own post on NNs that fo­cuses more on de­riv­ing back­prop from com­pu­ta­tional graphs. I think that method of do­ing so also builds up a lot of the Chain Rule in­tu­ition, as you can eas­ily see how the deriva­tives for ear­lier weights can be de­rived from those in later weights.

• lifelonglearner 13 Aug 2019 20:38 UTC

  4 points

  in reply to: Hazard's comment on: Hazard’s Short­form Feed

  I re­ally like that you’re do­ing this! I’ve tried to get into the se­ries, but I haven’t done so in a while. Thanks for the sum­maries!

  
  

  (Also, maybe it’d be good for fu­ture com­ments about what you’re do­ing to be chil­dren of this post, so it doesn’t break the flow of sum­maries.)

• lifelonglearner 8 Aug 2019 20:55 UTC

  4 points

  in reply to: Viliam's comment on: Owen Another Thing

  Your ad­vice about demon­strat­ing that you are ca­pa­ble alone is re­ally in­ter­est­ing. Thanks for the ex­tended ex­am­ples!

Cal­ibrat­ing With Cards

• lifelonglearner 8 Aug 2019 5:24 UTC

  5 points

  on: Owen Another Thing

  Ex­pe­rience As Com­pound­ing:

  Some­times I ask my­self: “A bunch of cool stuff seems to be hap­pen­ing in the pre­sent. So why can’t I move faster and let these things in? Why do I feel stuck by past things?”

  Well, ex­pe­rience com­pounds. One rea­son child­hood events can be so in­fluen­tial isn’t just that they hap­pened when you were at a for­ma­tive time and de­vel­op­ing your mod­els. In ad­di­tion, the fact that you pick them up early means they’ve had the priv­ilege of be­ing part of your thought pro­cesses for longer. They’re more well-worn tools.

  Then, there’s also the de­fault an­swer that each ad­di­tional year of your life is, rel­a­tive to the amount of years you’ve lived, a lesser amount. EX: From year 6 to 7, you’ve gained an ex­tra ~15% of your to­tal lifes­pan in new ex­pe­riences. Whereas from 26 to 27, you’ve gained closer to 4% of your to­tal lifes­pan in new ex­pe­riences.

  But, I’d like ev­ery year to be mea­sured more equally with one an­other. I feel like cool stuff is pass­ing by me right now, and I’m just slow on the up­take. I’m not tak­ing it in!

  Yes, you can get set in your older ways of think­ing, and you will have seen more with each suc­ces­sive year. But ex­pe­ri­en­tially speak­ing I’d like to get my brain to also pay more at­ten­tion to the re­cent stuff.

  I guess one hacky way to do this would be to spend more time ru­mi­nat­ing on the pre­sent (which is also harder be­cause if you’ve lived for 30 years, then by the same pro­por­tion­al­ity ar­gu­ment, there’s just less stuff to think about if you re­strict your­self to years 29-30).

  
  

  I’m con­fused be­cause there is also:

  
  

  Ex­pe­rience as a Slid­ing Win­dow:

  There’s some sort of cut­off point where I might be able to re­call things, but it no longer feels “re­cent” or di­rectly con­nected to my iden­tity.

  The feel­ing of re­cency is quite in­ter­est­ing to me be­cause it seems to im­ply that im­por­tant things are go­ing to fade over time. And if you want to pre­serve cer­tain parts of your iden­tity, there’s some sort of “up­keep” you’ll need to pay, i.e. hav­ing more of those sort of ex­pe­riences con­sis­tently so they stay in re­cent mem­ory.

  Any­way, that’s if you equate iden­tity with mem­ory, and that’s definitely an over­sim­plifi­ca­tion. But, what­ever.

  As new things filter in, older things drop out. I’m un­sure how to square this with the the­ory of com­pound­ing ex­pe­rience. Pre­sum­ably if some­thing has effects, even if it falls out of the win­dow, then things it in­fluenced can con­tinue to re­sound, ala dom­ino effect, but that feels quite con­trived. The ob­vi­ous an­swer, of course, is that there are sev­eral fac­tors at play.

• lifelonglearner 8 Aug 2019 5:21 UTC

  4 points

  on: Owen Another Thing

  One com­mon theme that I re­turn to, time and time again, is that of ad­dic­tive­ness. More speci­fi­cally, what makes some­thing habit-form­ing in a bad way? I’ve pre­vi­ously talked about this in the con­text of At­trac­tors. Lately, my thing to hate on is mo­bile games, or the thing that they rep­re­sent. Which, yes, is a lit­tle late to the game. And I don’t even play games on my mo­bile phone, so it seems a lit­tle out of place.

  But I digress. The point here is to talk about the Sk­in­ner Box. Or, the ap­pli­ca­tion of the same con­cept to hu­man things. Gam­ifi­ca­tion and no­tifi­ca­tion spam both fall into this cat­e­gory. But maybe not games. But maybe some games. Definitely mo­bile games. The point here is that there’s this cat­e­gory I want to get some clar­ity on, and it’s about these things which seem habit-form­ing and suck you in.

  So, what’s clearly a Sk­in­ner Box? I think that clicker games are to­tally Sk­in­ner Boxes. Also Clash of Clans, Far­mville (i.e. ev­ery­thing Zynga /​ Zynga-clones). But this line is of­ten hazy; Candy Box was in­no­va­tive and ex­cit­ing in cer­tain ways. There was a game a while back about al­pacas eat­ing one an­other that seemed sur­pris­ingly deep for an idle game. It’s one thing to put on a so­phis­ti­cated ve­neer on a game, but it still seems fine to cri­tique the un­der­ly­ing me­chan­ics.

  What does make a Sk­in­ner Box?

  1. Lack of a challenge

     1. De­spite hav­ing pro­gres­sion, idle and clicker games don’t re­ally have any­thing that forces the player to do any­thing strate­gic. They just...click things, and they get re­in­force­ment.

  2. In­stant gratification

     1. Mo­bile games of­ten lev­er­age this de­sire by time-lock­ing con­tent, prompt­ing you to pay in or­der to get some­thing now. The other thing to pay at­ten­tion here is if the feed­back loop is tight.

  3. In­cen­tives to keep go­ing?

     1. In­ter­mit­tent re­wards /​ re­ward schedules

  What doesn’t make a Sk­in­ner Box?

  1. Skill and growth

     1. The more some­thing is like an in­stru­ment or a sport, the less it seems like a Sk­in­ner Box. Although the many ca­sual LoL play­ers seem to in­di­cate that even some­thing which has a high skill cap can still be ad­dic­tive.

  2. Meaning

     1. The more you in­voke artis­tic pur­pose, nar­ra­tive, or some other agenda, we seem to be a lot more for­giv­ing about the ac­tual me­chan­ics in­volved.

  3. Instrumentality

     1. When we’re hun­gry, we eat and eat and eat. And no one bats an eye. The same thing with sleep. Stuff that’s use­ful isn’t of­ten seen as dan­ger­ous.

• lifelonglearner 8 Aug 2019 5:20 UTC

  4 points

  on: Owen Another Thing

  It feels like there’s been a push to­wards get­ting peo­ple to start cre­at­ing their own con­tent. Plat­forms like YouTube + the In­ter­net make it a lot eas­ier for peo­ple to start.

  Grow­ing an au­di­ence, though, seems hard be­cause there’s not of­ten a lot of free at­ten­tion. Most of the com­pe­ti­tion is zero-sum be­tween differ­ent con­tent. Peo­ple only have so much free time, so min­utes they spend en­gag­ing with your stuff is min­utes they don’t spend en­gag­ing in other peo­ple’s stuff.

  There’s a cyn­i­cal view­point here which is some­thing like “If you don’t think you’re cre­at­ing Good Con­tent, don’t broad­cast it! We have enough low-qual­ity stuff as it is, out there.”

  I think peo­ple of­ten want to cre­ate, though. It’s one of the de­fault re­sponses peo­ple have if you ask them “Say you could live com­fortably with­out need­ing to work. What would you do then?” (“Well, I’d write. Or I’d learn to play an in­stru­ment...”)

  Often, though, im­ple­men­ta­tion takes far more time than com­ing up with the ini­tial idea. There is an asym­me­try across many fields where the ac­tual ideation is done by only a small group of peo­ple. This then re­quires maybe 10X as many peo­ple to ac­tu­ally put into prac­tice. (EX: the peo­ple who de­sign the look/​feel of a piece of soft­ware at a com­pany vs those who build it.)

  Thus, if you want peo­ple to join your pro­ject (which is of course great be­cause you came up with it), you’ll need to con­vince other peo­ple to go with you. On the flip side, I think there’s a skill worth prac­tic­ing where you let go of idea own­er­ship. Stuff is go­ing to get done, and you’re go­ing to be do­ing it; who­ever came up with the idea might be less im­por­tant than whether or not you want the stuff to hap­pen.

  But maybe the de­sire for in­di­vi­d­ual ideation points to some­thing im­por­tant. A re­ally large amount of peo­ple seem to want to par­take in cre­ative en­deav­ors.
  

• lifelonglearner 8 Aug 2019 5:14 UTC

  4 points

  on: Owen Another Thing

  Here’s some­thing that feels like an­other in­stance of the de­on­tol­o­gist vs con­se­quen­tial­ist ab­strac­tion, ex­cept that the par­tic­u­lars of the situ­a­tion are what stick out to me: When I choose be­tween do­ing some­thing sane or some­thing that’s en­dorsed by an offi­cial rule, I’ll more-of­ten-than-I-like opt to do the en­dorsed thing, even when it’s ob­vi­ously worse-off for me.

  Some ex­am­ples, of vary­ing qual­ity:

  • Not jay­walk­ing, even when it’s in a neigh­bor­hood or oth­er­wise not-crowded place.

  • Ask­ing for per­mis­sion to do ob­vi­ous things in­stead of just do­ing them

  • Fo­cus­ing on the literal words that some­one ini­tially said, rather than their in­tent, or if they later re­cant.

  • Let­ting harm­ful poli­cies hap­pen in­stead of ap­peal­ing them.

  I’m re­minded of that study which showed that peo­ple fol­low­ing an evac­u­a­tion robot were led to stay in a room even when there was a fire, even when the robot was ob­served to be pre­vi­ously faulty. There’s some­thing about rules that over­rides ap­peal­ing to san­ity. I’m a lit­tle wor­ried that I bias to­wards this side com­pared to just do­ing the thing that works out bet­ter.

  There are of course benefits to choos­ing the offi­cial op­tion. The biggest one is that if some­one ques­tions your judg­ment later on, you can ap­peal to the es­tab­lished rules. That gives you a lot of so­cial back­ing to lean on.

  I think there’s also a weird masochis­tic as­pect of crav­ing pity, of want­ing to be in a situ­a­tion that seems bad by virtue of na­ture, so I can ab­solve my­self of re­spon­si­bil­ity. Some­thing about how this used to be a play to se­cure our­selves more re­sources, through a pity play?

• lifelonglearner 8 Aug 2019 5:13 UTC

  4 points

  on: Owen Another Thing

  Mal­colm Ocean gets it. There’s a ter­rible thing that hap­pens when you try to en­cap­su­late your es­say with a ti­tle. Some­how, the la­bel takes on a life of its own, and you some­times for­get the con­tent in­side the es­say.

  This hap­pens to my own es­says where I think “Oh, huh, this es­say is called ‘Learn­ing from Past Ex­pe­riences’”. Sounds kinda bor­ing.

  And in fact it was not bor­ing and it was good.

  I’m think­ing of maybe in­stead tran­si­tion­ing to just num­bers + sum­maries in­stead.

  For ex­am­ple, a for­mat like: Es­say 10 [Fad­ing nov­elty, ways to ad­dress it, and a brief di­gres­sion into ty­pog­ra­phy.]