# Mark Xu

Karma: 1,170

http://​​markxu.com

• This made me chuckle. More humor

• Ra­tion­al­ists tax­on­o­miz­ing rationalists

• Mesa-ra­tio­nal­ists (the mesa-op­ti­miz­ers in­side ra­tio­nal­ists)

• car­rier pi­geon rationalists

• proto-rationalists

• not-yet-born rationalists

• literal rats

• frequentists

• group-house rationalists

• EA fo­rum rationalists

• aca­demic rationalists

• meme rationalists

:)

• This is very ex­cit­ing. Look­ing for­ward to the rest of the se­quence.

As I was read­ing, I found my­self re­fram­ing a lot of things in terms of the rows and columns of the ma­trix. Here’s my loose at­tempt to red­erive most of the prop­er­ties un­der this view.

• The world is a set of states. One way to think about these states is by putting them in a ma­trix, which we call “carte­sian frame.” In this frame, the rows of the ma­trix are pos­si­ble “agents” and the columns are pos­si­ble “en­vi­ron­ments”.

• Note that you don’t have to put all the states in the ma­trix.

• En­surables are the part of the world that the agent can always en­sure we end up in. En­surables are the rows of the ma­trix, closed un­der supersets

• Preventa­bles are the part of the world that the agent can always en­sure we don’t end up in. Preventa­bles are the com­ple­ments of the rows, closed un­der subsets

• Con­trol­lables are parts of the world that are both en­surable and pre­ventable. Con­tro­lables are rows (or sets of rows) for which there ex­ists rows that are dis­joint. [edit: pre­vi­ous defi­ni­tion of “con­tains el­e­ments not found in other rows” was wrong, see com­ment by crab­man]

• Ob­serve­ables are parts of the en­vi­ron­ment that the agent can ob­serve and act con­di­tion­ally ac­cord­ing to. Ob­serv­ables are columns such that for ev­ery pair of rows there is a third row that equals the 1st row if the en­vi­ron­ment is in that column and the 2nd row oth­er­wise. This means that for ev­ery two rows, there’s a third row that’s made by tak­ing the first row and swap­ping el­e­ments with the 2nd row where it in­ter­sects with the column.

• Ob­serv­ables have to be sets of columns be­cause if they weren’t, you can find a column that is par­tially ob­serv­able and par­tially not. This means you can build an ac­tion that says some­thing like “if I am ob­serv­able, then I am not ob­serv­able. If I am not ob­serv­able, I am ob­serv­able” be­cause the swap­ping doesn’t work prop­erly.

• Ob­serv­ables are closed un­der boolean com­bi­na­tion (note it’s suffi­cient to show clo­sure un­der com­ple­ment and unions):

• Since swap­ping in­dex 1 of a row is the same as swap­ping all non-1 in­dexes, ob­serv­ables are closed un­der com­ple­ments.

• Since you can swap in­dexes 1 and 2 by first swap­ping in­dex 1, then swap­ping in­dex 2, ob­serv­ables are closed un­der union.

• This is equiv­a­lent to say­ing “If A or B, then a0, else a2” is log­i­cally equiv­a­lent to “if A, then a0, else (if B, then a0, else a2)”

• Since con­trol­lables are rows with spe­cific prop­er­ties and ob­serv­ables are columns with spe­cific prop­er­ties, then noth­ing can be both con­trol­lable and ob­serv­able. (The only pos­si­bil­ity is the en­tire ma­trix, which is triv­ially not con­trol­lable be­cause it’s not pre­ventable)

• This as­sumes that the ma­trix has at least one column

• The image of a carte­sian frame is the ac­tual ma­trix part.

• Since an en­surable is a row (or su­per­set) and an ob­serv­able is a column (or set of columns), then if some­thing is en­surable and ob­serv­able, then it must con­tain ev­ery column, so it must be the whole ma­trix (image).

• If the ma­trix has 1 or 0 rows, then the ob­serv­able con­straint is triv­ially satis­fied, so the ob­serv­ables are all pos­si­ble sets of (pos­si­ble) en­vi­ron­ment states (since 01 length columns are the same as states).

• “0 rows” doesn’t quite make sense, but just pre­tend that you can have a 0 row ma­trix which is just a set of world states.

• If the ma­trix has 0 columns, then the en­surable/​pre­ventable con­traint is triv­ially satis­fied, so the en­surables are the same as the pre­venta­bles are the same as the con­trol­lables, which are all pos­si­ble sets of (pos­si­ble) en­vi­ron­ment states (since “length 0” rows are the same as states).

• “0 columns doesn’t make that much sense ei­ther but pre­tend that you can have a 0 column ma­trix which is just a set of world state.

• If the ma­trix has ex­actly 1 column, then the en­surable/​pre­ventable con­straint is triv­ially satis­fied for states in the image (ma­trix), so the en­surables are all non-empty sets of states in the ma­trix (since length 1 columns are the same as states), closed un­der union with states out­side the ma­trix. It should be easy to see that con­trol­lables are all pos­si­ble sets of states that in­ter­sect the ma­trix non-triv­ially, closed un­der union with states out­side the ma­trix.

• In 4.1:

Given a0 and a1, since S∈Obs(C), there ex­ists an a2∈A such that for all e∈E, we have a2∈if(S,a0,a1). Then, since T∈Obs(C), there ex­ists an a3∈A such that for all e∈E, we have a3∈if(S,a0,a2). Un­pack­ing and com­bin­ing these, we get for all e∈E, a3∈if(S∪T,a0,a1). Since we could con­struct such an a3 from an ar­bi­trary a0,a1∈A, we know that S∪T∈Obs(C). □

I think there’s a typo here. Should be , not .

(also not sure how to copy la­tex prop­erly).

• prob­lem: I don’t do enough fo­cused work in a day.

1. set aside set times for fo­cused work via calendar

2. put “do fo­cused work” on my todo list (ac­tu­ally already did this and worked sur­pris­ingly well for a week—why doesn’t it work as well any­more?)

3. block var­i­ous chat­ting apps

4. block less­wrong?

5. do pomodoros

6. use some cowork­ing space to en­courage focus

7. take more breaks

8. eat healthier food (pos­si­bly no carbs) to have more energy

9. get a bet­ter sleep sched­ule to have more energy

10. med­i­tate more for bet­ter meta-cog­ni­tion and focus

11. try to do de­liber­ate prac­tice on do­ing fo­cused work

12. in­stall a num­ber of TAPs re­lated to sup­press­ing de­sires for dis­trac­tion, e.g. “im­pulse to stop work­ing → check po­modoro timer”

13. I’m told com­plice is useful

14. daily re­views might be helpful?

15. be more spe­cific when do­ing weekly review

16. make more com­mit­ments to other peo­ple about the amount of out­put I’m go­ing to have, cre­at­ing so­cial pres­sure to ac­tu­ally pro­duce that amount of output

17. be more care­ful when schedul­ing calls with peo­ple so i have long se­ries of un­in­ter­rupted hours

18. take more naps when I no­tice I’m los­ing focus

19. be more re­al­is­tic about the amount of fo­cused work I can do in a day (does “re­al­ize this isn’t ac­tu­ally a prob­lem” count as solv­ing it? seems like yes)

20. vary the length of pomodoros

21. do re­solve cy­cles for solu­tions to the prob­lem, im­ple­ment­ing some of them

22. read var­i­ous pro­duc­tivity books, like the pro­cras­ti­na­tion equa­tion, GTD, tiny habits, etc.

23. ex­er­cise more for more en­ergy (un­for­tu­nately, the mind is cur­rently still em­bod­ied)

24. make sure I’m fo­cus­ing on the right things—bet­ter to spend half the time fo­cus­ing on the most im­por­tant thing than dou­ble the time on the 2nd most im­por­tant thing

25. spend more time work­ing with people

26. stop filling non-work time with ac­tivi­ties that cause men­tal fa­tigue, like read­ing, pod­casts, etc.

27. stop do­ing mis­cel­la­neous things from my todolist dur­ing “breaks”, e.g. don’t do laun­dry be­tween po­modoros, just lie on the floor and rest

28. get into a bet­ter rhythm of work/​break cy­cles, e.g. treat ev­ery hour as a con­tigu­ous block by de­fault, schedul­ing calls on hour de­mar­ca­tions only

29. use lap­top in­stead of large mon­i­tor—large screens might make it eas­ier to get distracted

30. block the in­ter­net on my com­puter dur­ing cer­tain pe­ri­ods of time so I can fo­cus on writing

31. take var­i­ous drugs that give me more en­ergy, e.g. caf­feine, nico­tine, and other substances

32. stop drink­ing things like tea—the caf­feine might give more en­ergy, but make fo­cus­ing harder

33. wear noise-can­cel­ing head­phones to block out dis­trac­tions from noise

34. listen to mu­sic de­signed to en­courage fo­cus, like cool rhythms or video game music

35. work on things that are ex­cit­ing—fo­cus isn’t a prob­lem if they’re in­trin­si­cally enjoyable

36. Ben Kuhn has some good tips—check those out again

37. Res­cueTime says most of my dis­tract­ing time is on mes­sen­ger and sig­nal. I think quaran­tine is mess­ing with my de­sire for so­cial in­ter­ac­tion. Figure out how to re­place that some­how?

39. make sure to have snacks to keep up blood sugar

40. al­ter­nate be­tween stand­ing desk and sit­ting desk to add novelty

41. re­duce cost for start­ing to do fo­cused work by hav­ing a clear list of fo­cused work that needs to be done, leav­ing com­puter in state ready to start im­me­di­ately upon com­ing back to it

42. nudge my­self into do­ing fo­cused work by do­ing tasks that re­quire micro-fo­cus first, like make metac­u­lus pre­dic­tions, then move on to more im­por­tant fo­cused work

44. work on more phys­i­cal sub­strates, e.g. pa­per+pen, whiteboard

45. use a non-linux op­er­at­ing sys­tem to get ac­cess to bet­ter tools for fo­cus­ing, like cold turkey, free­dom, etc.

46. switch mouse to left hand which will cause more effort to be needed to mind­lessly use com­puter, po­ten­tially de­creas­ing mindlessness

47. ac­quire more desk­toys to serve as non-com­puter dis­trac­tions that might pre­serve fo­cus better

48. prac­tice fo­cus­ing on non-work thing, e.g. by study­ing a ran­dom sub­ject, play­ing a game I don’t like, be­ing more mind­ful in ev­ery­day life, etc.

49. do more yoga to feel more pre­sent in body

50. TAP com­mon idle ac­tivity I do with “fo­cus on work”, e.g. crack knuck­les, stretch arms, ad­just seat.

Time taken: 20 minutes

More things I thought of af­ter read­ing Ra­fael Harth’s re­sponse:

1. use some­thing like bee­minder to do more fo­cused work

2. do re­search di­rectly into what causes some peo­ple to be bet­ter at fo­cus­ing than others

3. ask peo­ple that seem to be good at do­ing fo­cused work for tips

4. reread Deep Work and take it more seriously

• I per­son­ally see no fun­da­men­tal differ­ence be­tween di­rect and in­di­rect ways of in­fluence, ex­cept in so far as they re­late to stuff like ex­pected value.

I agree that given the amount ex­pected in­fluence, other uni­verses are not high on my pri­or­ity list, but they are still on my pri­or­ity list. I ex­pect the same for con­se­quen­tial­ists in other uni­verses. I also ex­pect con­se­quen­tial­ist be­ings that con­trol most of their uni­verse to get around to most of the things on their pri­or­ity list, hence I ex­pect them to in­fluence the Sol­monoff prior.

• Con­se­quen­tial­ists can rea­son about situ­a­tions in which other be­ings make im­por­tant de­ci­sions us­ing the Solomonoff prior. If the mul­ti­ple be­ings are simu­lated them, they can de­cide ran­domly (be­cause hav­ing e.g. 1100 of the re­sources is bet­ter than none, which is the ex­pec­ta­tion of “blind mischievous­ness”).

An ex­am­ple of this sort of rea­son­ing is New­comb’s prob­lem with the knowl­edge that Omega is simu­lat­ing you. You get to “con­trol” the re­sult of your simu­la­tion by con­trol­ling how you act, so you can in­fluence whether or not Omega ex­pects you to one-box or two-box, con­trol­ling whether there is \$1,000,000 in one of the boxes.

• My cur­rent tax­on­omy of ra­tio­nal­ists is:

• LW ra­tio­nal­ists (HI!)

• Face­book rationalists

• Twit­ter rationalists

• Blog rationalists

• In­ter­net-in­visi­ble rationalists

Are there other types of ra­tio­nal­ists? Maybe like group-chat ra­tio­nal­ists? or pod­cast ra­tio­nal­ists? google doc ra­tio­nal­ists?

• An in­tu­itive ex­pla­na­tion of the kelly crite­rion, with a bunch of worked ex­am­ples. Zvi’s post is good but lacks worked ex­am­ples and jus­tifi­ca­tion for heuris­tics. Ja­co­bian ad­vises us to Kelly bet on ev­ery­thing, but I don’t un­der­stand what a “kelly bet” is in all but the sim­plest fi­nan­cial sce­nar­ios.

• I want more peo­ple to write down their mod­els for var­i­ous things. For ex­am­ple, a model I have of the econ­omy is that it’s a bunch of boxes with in­puts and out­puts that form a sparsely di­rected graph. The length of the short­est cy­cle con­trols things like eco­nomic growth and AI take­off speeds.

Another ex­am­ple is that peo­ple have work­ing mem­ory in both their brains and their bod­ies. When their brain-work­ing-mem­ory is full, in­for­ma­tion gets stored in their bod­ies. Tech­niques like fo­cus­ing are of­ten use­ful to ex­tact in­for­ma­tion stored in body-work­ing-mem­ory.

• A min­i­mal-as­sump­tion de­scrip­tion of Up­date­less De­ci­sion The­ory. This wiki page de­scribes the ba­sic con­cept, but doesn’t in­clude mo­ti­va­tion, ex­am­ples or in­tu­ition.

• A thor­ough de­scrip­tion of how to do pair de­bug­ging, a CFAR ex­er­cise par­tially de­scribed here.

• A solid, min­i­mal-as­sump­tion de­scrip­tion of value hand­shakes. This SSC post con­tains the best de­scrip­tion of which I’m aware, which I think is slightly sad:

Values hand­shakes are a pro­posed form of trade be­tween su­per­in­tel­li­gences. Sup­pose that hu­mans make an AI which wants to con­vert the uni­verse into pa­per­clips. And sup­pose that aliens in the An­dromeda Galaxy make an AI which wants to con­vert the uni­verse into thumb­tacks.

When they meet in the mid­dle, they might be tempted to fight for the fate of the galaxy. But this has many dis­ad­van­tages. First, there’s the usual risk of los­ing and be­ing wiped out com­pletely. Se­cond, there’s the usual dead­weight loss of war, de­vot­ing re­sources to mil­i­tary buildup in­stead of pa­per­clip pro­duc­tion or what­ever. Third, there’s the risk of a Pyrrhic vic­tory that leaves you weak­ened and easy prey for some third party. Fourth, no­body knows what kind of scorched-earth strat­egy a los­ing su­per­in­tel­li­gence might be able to use to thwart its con­queror, but it could po­ten­tially be re­ally bad – eg ini­ti­at­ing vac­uum col­lapse and de­stroy­ing the uni­verse. Also, since both par­ties would have su­per­in­tel­li­gent pre­dic­tion abil­ities, they might both know who would win the war and how be­fore ac­tu­ally fight­ing. This would make the fight­ing re­dun­dant and kind of stupid.

Although they would have the usual peace treaty op­tions, like giv­ing half the uni­verse to each of them, su­per­in­tel­li­gences that trusted each other would have an ad­di­tional, more at­trac­tive op­tion. They could merge into a su­per­in­tel­li­gence that shared the val­ues of both par­ent in­tel­li­gences in pro­por­tion to their strength (or chance of mil­i­tary vic­tory, or what­ever). So if there’s a 60% chance our AI would win, and a 40% chance their AI would win, and both AIs know and agree on these odds, they might both rewrite their own pro­gram­ming with that of a pre­vi­ously-agreed-upon child su­per­in­tel­li­gence try­ing to con­vert the uni­verse to pa­per­clips and thumb­tacks in a 60-40 mix.

This has a lot of ad­van­tages over the half-the-uni­verse-each treaty pro­posal. For one thing, if some re­sources were bet­ter for mak­ing pa­per­clips, and oth­ers for mak­ing thumb­tacks, both AIs could use all their re­sources max­i­mally effi­ciently with­out hav­ing to trade. And if they were ever threat­ened by a third party, they would be able to pre­sent a com­pletely unified front.

# [Question] What posts do you want writ­ten?

19 Oct 2020 3:00 UTC
46 points
• Less­wrong posts that I want some­one to write:

1. De­scrip­tion of pair debugging

2. De­scrip­tion of value handshakes

Maybe I’ll think of more later.

• The S. prior is a gen­eral-pur­pose prior which we can ap­ply to any prob­lem. The out­put string has no mean­ing ex­cept in a par­tic­u­lar ap­pli­ca­tion and rep­re­sen­ta­tion, so it seems sense­less to try to in­fluence the prior for a string when you don’t know how that string will be in­ter­preted.

The claim is that con­se­quen­tal­ists in simu­lated uni­verses will model de­ci­sions based on the Solomonoff prior, so they will know how that string will be in­ter­preted.

Can you give an in­stance of an ap­pli­ca­tion of the S. prior in which, if ev­ery­thing you wrote were cor­rect, it would mat­ter?

Any de­ci­sion that con­trols sub­stan­tial re­source al­lo­ca­tion will do. For ex­am­ple, if we’re evalut­ing the im­pact of run­ning var­i­ous pro­grams, blow up planets, in­terfere will alien life, etc.

Also in the cat­e­gory of “it’s a fea­ture, not a bug” is that, if you want your val­ues to be right, and there’s a way of learn­ing the val­ues of agents in many pos­si­ble uni­verses, you ought to try to figure out what their val­ues are, and up­date to­wards them. This ar­gu­ment im­plies that you can get that for free by us­ing Solomonoff pri­ors.

If you are a moral re­al­ist, this does seem like a pos­si­ble fea­ture of the Solomonoff prior.

Third, what do you mean by “the out­put” of a pro­gram that simu­lates a uni­verse?

A TM that simu­lates a uni­verse must also spec­ify an out­put chan­nel.

Take your ex­am­ple of Life—is the out­put a raster scan of the 2D bit ar­ray left when the uni­verse goes static? In that case, agents have lit­tle con­trol over the ter­mi­nal state of their uni­verse (and also, in the case of Life, the string will be ei­ther al­most en­tirely ze­roes, or al­most en­tirely 1s, and those both already have huge Solomonoff pri­ors). Or is it the con­cate­na­tion of all of the states it goes through, from start to finish?

All of the above. We are run­ning all pos­si­ble TMs, so all com­putable uni­verses will be paired will all com­putable out­put chan­nels. It’s just a ques­tion of com­plex­ity.

Are you imag­in­ing that bits are never out­put un­less the ac­ci­den­tally-simu­lated aliens choose to out­put a bit? I can’t imag­ine any way that could hap­pen, at least not if the uni­verse is speci­fied with a short in­struc­tion string.

No.

This brings us to the 4th prob­lem: It makes lit­tle sense to me to worry about av­er­ag­ing in out­puts from even mere plane­tary simu­la­tions if your com­puter is just the size of a planet, be­cause it won’t even have enough mem­ory to read in a sin­gle out­put string from most such simu­la­tions.

I agree that ap­prox­i­ma­tion the Sol­monoff prior is difficult and thus its ma­lig­nancy prob­a­bly doesn’t mat­ter in prac­tice. I do think similar ar­gu­ments ap­ply to cases that do mat­ter.

5th, you can weigh each pro­gram’s out­put pro­por­tional to 2^-T, where T is the num­ber of steps it takes the TM to ter­mi­nate. You’ve got to do some­thing like that any­way, be­cause you can’t run TMs to com­ple­tion one af­ter an­other; you’ve got to do some­thing like take a large ran­dom sam­ple of TMs and iter­a­tively run each one step. Prob­lem solved.

See the sec­tion on the Speed prior.

Per­haps the biggest prob­lem is that you’re talk­ing about an en­tire uni­verse of in­tel­li­gent agents con­spiring to change the “out­put string” of the TM that they’re run­ning in. This re­quires them to re­al­ize that they’re run­ning in a simu­la­tion, and that the out­put string they’re try­ing to in­fluence won’t even be looked at un­til they’re all dead and gone. That doesn’t seem to give them much mo­ti­va­tion to de­vote their en­tire civ­i­liza­tion to twid­dling bits in their uni­verse’s fi­nal out­put in or­der to shift our pri­ors in­finites­i­mally. And if it did, the more likely out­come would be an in­ter­galac­tic war over what string to out­put.

They don’t have to re­al­ize they’re in a simu­la­tion, they just have to re­al­ize their uni­verse is com­putable. Con­se­quen­tial­ists care about their val­ues af­ter they’re dead. The cost of in­flunc­ing the prior might not be that high be­cause they only have to com­pute it once and the benefit might be enor­mous. Ex­po­nen­tial de­cay + acausal trade make an in­ter­galac­tic war un­likely.