[Old] Mapmaking Series

A few years ago I had a differ­ent blog than my cur­rent one and I wrote two se­ries of posts there. I no longer think they are es­pe­cially good, but thanks to my re­cent link­ing of the sec­ond se­ries in com­ments on a dis­cus­sion here on LW, I have got­ten feed­back that they were use­ful to a cou­ple peo­ple. So in the spirit of mak­ing this con­tent a lit­tle more ac­cessible, eas­ily find­able, and link­able, I’m re­post­ing them here with min­i­mal edit­ing. I’m group­ing them to­gether in two posts to cover the two se­ries of posts I made.

This first se­ries is about LW ra­tio­nal­ity ba­sics largely cov­ered in the se­quences, but I wanted to ex­press those ideas in my own words. It was mostly use­ful for me to do that work of putting my un­der­stand­ing into words, and I’m not sure how much read­ing the se­ries in this post is worth­while to oth­ers, but the sec­ond se­ries refer­ences it, so seems worth hav­ing it here for ease of ac­cess. The sec­ond se­ries is prob­a­bly more valuable, and I’ll link it here once it’s up. ETA: It’s up.

map and territory

let’s start at the be­gin­ning. both my be­gin­nings with ra­tio­nal­ity and the first place you should start to find the win­ning way.

by now the metaphor seems well worn to me, but i have to re­mem­ber that there was a time when i didn’t know about it. a time when i could get con­fused about the cor­re­spon­dence be­tween my own thoughts and re­al­ity. and even a time when i had the words to sep­a­rate the two but could still get things con­fused. so let me try to ease you in to our first at­tempt to break and rein­te­grate your think­ing: the metaphor of the map and the ter­ri­tory.

a wise king com­mands a map of perfect de­tail be made of his king­dom. the map mak­ers be­gin the pro­ject. an­i­mals in their own and neigh­bor­ing king­doms are slaugh­tered to make the parch­ment. forests are cleared to col­lect all their resin and gi­ant holes are left in the ground from ex­tract­ing graphite to make the ink. the army of map mak­ers works fu­ri­ously day and night un­til at last they tell the king the pro­ject is finished. in the morn­ing, they say, the map shall be re­vealed to him.
when the king wakes up he no­tices it’s un­usu­ally dark. look­ing out the royal win­dow, he sees that the sky has turned a yel­low­ish tan and the light is blocked out. he stares at the sky, ex­am­in­ing the lines that seem to have re­placed the sun and clouds. slowly re­al­iza­tion dawns on him: the sky has been re­placed by a map!
the king runs down­stairs, still in his night­gown, call­ing for his map mak­ers. the king throws open the doors of the throne room to find them gath­ered, nod­ding and smil­ing to them­selves.
“what is the mean­ing of this?!?” he bel­lows.
“why, sire,” the head map maker says, “we have com­pleted the pro­ject as re­quested. you need now only look up to see a perfectly ac­cu­rate map of the king­dom in the sky.”

it’s easy, in hind­sight, to see the king’s con­fu­sion. in ask­ing for a perfectly ac­cu­rate map, he got the only thing that could be perfectly ac­cu­rate: an ex­act re­flec­tion of the king­dom. in fact, in this story, i’m dis­ap­pointed the map mak­ers didn’t man­age to achieve greater ac­cu­racy by not con­struct­ing a map at all and just declar­ing the king­dom to be the map of it­self. to liter­ally make the map the ter­ri­tory.

when we talk of maps what we re­ally want are smaller ver­sions of re­al­ity that we can carry in our pock­ets. or, less metaphor­i­cally speak­ing, thoughts we can use to model the world.

this is one of the things folks mean when they say that the map is not the ter­ri­tory: the map serves a differ­ent pur­pose than the ter­ri­tory, and it would be im­prac­ti­cal to use the ter­ri­tory as a map. or, to again the break down the metaphor, mod­els are use­ful be­cause they are thoughts we use to rea­son about the state of the world. there can be no such thing as a perfect model: all mod­els con­tain er­ror and un­cer­tainty. you can­not be 100% or 0% sure of some­thing. prob­a­bil­ity is sub­jec­tive; be­liefs con­tain un­cer­tainty.

maps are simple

in­her­ent in the map and ter­ri­tory anal­ogy is a cer­tain con­cep­tion of what ‘truth’ means. it re­lies on what some might call the sim­ple truth or the naive per­spec­tive on re­al­ity. its points are this:

  • the world ex­ists and is uni­ver­sal (the same for ev­ery­one)

  • ev­ery­one per­ceives the world for them­selves (we each make our own map)

  • the qual­ity of the map is a mea­sure of how well it shows (pre­dicts) the lo­ca­tions (states) of the world

an­other way to say this is: the truth is out there, and it’s your job to map it.

this is worth point­ing out be­cause for the last cen­tury there’s been a trend in west­ern think­ing to con­flate the map with the ter­ri­tory. it pops up in the­o­ries like “truth is rel­a­tive” and “re­al­ity is made by con­sen­sus”. sure enough, since maps ex­ist they are part of the ter­ri­tory, but this does not mean the map is the ter­ri­tory. maps can be built col­lab­o­ra­tively, and we all have our own maps, but this doesn’t make truth col­lab­o­ra­tive or rel­a­tive: it makes map mak­ing a shared task and maps sub­jec­tive.

the fol­low­ing short story may help make this clearer. my goal is to cause, if it has not already hap­pened for you, a shift in the way you un­der­stand your re­la­tion­ship with re­al­ity. it hap­pened for me when i read cfai sec­tion 3.1. i hope it will or already has hap­pened for you, be­cause fully un­der­stand­ing the re­la­tion­ship be­tween the map and the ter­ri­tory is es­sen­tial to mak­ing progress in the win­ning way.

For gen­er­a­tions, the Shep­herd had been tasked with keep­ing track of the village’s flock. Every day he let the sheep out to graze and then re­called them all at night to the safety of the fence. The duty passed from father to son, re­quiring years of train­ing to be able to re­mem­ber all the sheep and no­tice if one went miss­ing.
The last Shep­herd, known to his friends as Artzain, was es­pe­cially skil­led. He could rec­og­nize all his sheep and knew when one was miss­ing. He never failed to no­tice a miss­ing sheep, even if he couldn’t always find the miss­ing ones be­fore night fell and the wolves preyed on them.
Artzain’s son, Umalusi, seemed des­tined to lose much of the flock. Although Artzain spent many hours ev­ery day teach­ing Umalusi how to rec­og­nize the sheep, to tell them apart by sub­tle differ­ences, and no­tice if one was miss­ing, Umalusi could no bet­ter tell apart the sheep than any other villager.
Dis­heart­ened, Artzain stopped ac­tively teach­ing Umalusi. He com­mit­ted him­self to the idea that he would have to live long enough to train his grand­son, lest Umalusi lose all the sheep and the village starve.
Left to him­self, Umalusi spent his time play­ing games and mak­ing up new, ever more cre­ative games to oc­cupy his time. This seemed not very use­ful, but since the only job available to Umalusi was Shep­herd, and Artzain had no in­ten­tion of let­ting him mind the flock, ev­ery­one left him alone with his games.
As Umalusi en­tered his 16th year, his father fell sud­denly ill. Artzain was un­able to stand, and so could not mind the flock. He had no choice but to send out Umalusi to act as Shep­herd.
When he came home on the first night, Artzain asked Umalusi if he had lost any sheep.
“No, father. They all re­turned to the fence,” said Umalusi.
Artzain sighed. To the un­trained eye it prob­a­bly did look like they had all re­turned, but in his heart he knew that his son had prob­a­bly lost a few sheep to the wolves.
The next day he was still too weak, so Artzain sent Umalusi in his place again. And again, Umalusi came home and re­ported hav­ing lost no sheep, and again Artzain didn’t re­ally be­lieve him.
After three more days of Umalusi watch­ing the sheep and claiming that none had been lost, Artzain re­gained his strength and was able to shep­herd again. On his first morn­ing back with his flock, he was as­ton­ished to find that all of the sheep were there. He hol­lered for Umalusi, who came run­ning think­ing some­thing ter­rible had hap­pened to his father.
“What’s wrong?” said Umalusi.
“No sheep are miss­ing,” said Artzain. “How?”
“What do you mean ‘stones’?”“You have already let the sheep out for to­day, so I can’t show you, but let me mind the sheep to­mor­row and I’ll show you.”
The next morn­ing, Artzain and Umalusi went to the sheep en­clo­sure to­gether as they had not done since Umalusi was a child. Artzain ex­tended his arm to tell Umalusi to pro­ceed, and Umalusi walked over to the gate.
There, he set down two large clay pots. One was filled with river peb­bles, and the other was empty. He opened the gate just wide enough that one sheep could make it through at a time. As the sheep moved out of the pad­dock and into the pas­ture, he be­gan mov­ing stones from one clay pot to the other.
“What are you do­ing, my boy,” said Artzain.
“Count­ing sheep,” said Umalusi.
“What is count­ing?”
“It’s a game I made up for track­ing things. See, when a sheep leaves the pad­dock, I move a stone from the left pot into the right.”
Umalusi paused to move some stones from the left pot to the right be­cause some sheep had gone out of the pad­dock while he was talk­ing.
“I know you like games, son, but what has that got to do with sheep?” asked Artzain.
“Imag­ine that each sheep is a stone. The left pot is the pad­dock and the right pot is the pas­ture. I use the stones to know where the sheep are,” said Umalusi.
“Ah, I see,” said Artzain, “you have se­cretly been study­ing magic from Shaman while I have been out with the sheep all day. You have learned some spell to con­trol the sheep with the stones.”
“Not at all,” said Umalusi, plop­ping a few more stones from the left pot to the right. “I know noth­ing of magic.”
“Then how can you be con­trol­ling the sheep with the stones?”
“I’m not. I move the stones when the sheep move.”
Artzain chuck­led heartily. “I see. So you ex­pect me to be­lieve, then, that the sheep have put a spell on you, and that they are mak­ing you move the stones rather than the other way around!”
“In a way, that’s true,” said Umalusi. “But the sheep have no mag­i­cal power over me. I move the stones to match where the sheep are. I only win if there is one stone for each sheep in the cor­rect pot”
“Fine, but how do you know which stone be­longs to which sheep? Now you have to be able to tell apart all the sheep and all the stones.”
“Nope. To me, all the sheep look the same. All the stones look the same, too. So it doesn’t mat­ter to me which sheep or which stone I move, so long as there are as many stones as there are sheep.”
“But no two sheep are the same,” said Artzain.
“And no two stones are the same,” said Umalusi.“
So how can you pos­si­bly say it doesn’t mat­ter so long as you have the right num­ber of stones in the right pots?”
“Be­cause al­though all sheep are differ­ent and all stones are differ­ent, I can treat them all the same in my game. So to me one sheep in the pad­dock is the same as any other sheep in the pad­dock, so one stone in the left pot is the same as any other stone in the left pot.”
Artzain stood silent and thought for a few min­utes. Fi­nally he said, “I think I un­der­stand, but to me it seems strange to ig­nore de­tails in or­der to be able to do some­thing you couldn’t do with them. Yet I can’t deny the fact that you’ve man­aged not to lose any sheep, even though you can’t tell them apart, by do­ing this.”
“I ad­mit, I also found it a bit odd at first, and if I’m hon­est I’ve been se­cretly test­ing this method out from a dis­tance while you watched the flock. When you got sick, it was my first chance to try it for real.”
“Well, I’m glad you suc­ceeded. Now I can re­tire. You are now Shep­ard.”
“Thanks, but I’m not sure I want to stay Shep­ard. I have big­ger games to play.”

maps make meaning

when i talk about the map and the ter­ri­tory, i talk al­most ex­clu­sively about the map, and when i do men­tion the ter­ri­tory it’s as some un­differ­en­ti­ated soup that ex­ists ‘out there’. turns about, that’s be­cause we liter­ally can­not talk about any­thing other than maps.

there is a fa­mous paint­ing of a pipe with the cap­tion “this is not a pipe”. on the sur­face this may seem odd: you are most definitely look­ing at a pipe. but re­ally the thing you see is not a pipe, but rather a rep­re­sen­ta­tion of one. that is to say, a map.

let that sink in a bit. we say ‘the map is not the ter­ri­tory’, but here we are be­ing quite ex­plicit about it. the map of a thing is not the thing it­self.

once you un­der­stand this, you are led to the in­evitable con­clu­sion that you have never di­rectly known re­al­ity in your life.

since pipes are rel­a­tively rare now, con­sider the hum­ble spoon. you may ob­ject that you’ve seen spoons in real life, held them in your hand, maybe even used them to eat. surely that was the ter­ri­tory. but con­sider, how did you even know that thing was a spoon, or even a thing sep­a­rate from other things? how could you tell that some small part of the uni­verse was dis­tinct from the rest of it in a way that you might call that part of it a spoon? be­cause you have a map that al­lows you to liter­ally make sense of things: to tell stuff in the uni­verse apart.

to put it an­other way, there is no spoon un­til you cre­ate it. as far as the uni­verse is con­cerned (and it’s not con­cerned with any­thing at all in such a fun­da­men­tal way you can’t even re­ally say the uni­verse is or is not con­cerned), spoons do not ex­ist. the ter­ri­tory is just stuff that has par­tic­u­lar prop­er­ties. al­though the stuff you will soon call a spoon ex­ists in­de­pen­dent of your un­der­stand­ing of it, it does not re­ally be­come a spoon un­til it is mapped into one. the map makes the mean­ing, and it is only in the map that the spoon ap­pears out of re­al­ity’s con­founded state.

in fact, there is no ter­ri­tory with­out the map, be­cause the sep­a­ra­tion of map and ter­ri­tory is it­self part of the map. with­out the map, there would be noth­ing to call ter­ri­tory in con­trast to it: it would sim­ply be ev­ery­thing, with all its com­plex­ities and yet none of its differ­en­ti­a­tion. it is the map that makes the mean­ing be­cause the ter­ri­tory is con­fused. not in the usual sense of ‘con­fused’, but in the literal sense based on it’s et­y­mol­ogy: all mixed to­gether as to be in­dis­t­in­guish­able. the ter­ri­tory is con­fused, and you will be con­fused with­out the map to help you make mean­ing of it.

maps are wrong

if you have ever used a ge­o­graph­i­cal map to help you nav­i­gate roads or the wilds or the ocean, you have in­evitably en­coun­tered dis­agree­ment be­tween the map and the ter­ri­tory you are in: the road you are look­ing for isn’t there, a boulder is block­ing your way, or the wa­ter is much shal­lower than re­ported. in these ways maps can be wrong, and yet we wouldn’t say the map is all wrong: if noth­ing else it was right enough that you made it as far as find­ing the map’s er­rors. when we say ‘the map is wrong’ what we re­ally mean is ‘it has er­rors’. un­sur­pris­ingly, this is also true of our metaphor­i­cal maps of the ter­ri­tory of re­al­ity.

be­cause the map is not the ter­ri­tory, it must con­tain er­rors. this is sup­ported by our knowl­edge of ther­mo­dy­nam­ics and its math­e­mat­i­cal ab­strac­tion, in­for­ma­tion the­ory, with as­sis­tance from spe­cial rel­a­tivity to pre­vent any­thing hap­pen­ing in­stan­ta­neously. and don’t try any quan­tum trick­ery: at most it just shows we can move bits around be­fore cre­at­ing them. so if we can­not have an er­ror-free map, we should con­sider how to live with those er­rors.

al­though i like to talk about maps, in ev­ery­day lan­guage peo­ple talk about be­liefs, i.e. thoughts about the state of the world. and the naive model of be­lief com­mon in the oc­ci­dent is that be­liefs are ei­ther true (perfectly pre­dict re­al­ity) or false. this leads some folks to talk of re­ject­ing all be­liefs, es­pe­cially be­cause ‘be­lief’ is of­ten a word as­so­ci­ated with re­li­gion. while this po­si­tion is un­der­stand­able given a true or false model of be­lief, from a tech­ni­cal stand­point, be­lief is the right term, so what we need is a bet­ter the­ory of be­lief.

that is to say, we need a more use­ful map of the map.

the best model i know of is to treat be­liefs as sub­jec­tive and prob­a­bil­is­tic. if you think of a be­lief as a point on the map, each be­lief is made up of at least two parts: a state­ment about re­al­ity and a like­li­hood of cor­rect­ness. this ex­pands the no­tion of cor­rect­ness as­sumed by the naive model of be­lief by adding a mechanism for as­sess­ing the maybe-ness of a be­lief. with this richer model of be­lief, we can un­der­stand the map as some­thing that may be mostly right, but not in the sense of mostly made up of true things, but in the sense of mostly made up of things that are mostly likely to be true. the bur­den of truth shifts from the map to the ter­ri­tory, where it be­longs, and leaves the map with the re­spon­si­bil­ity of as­sess­ing its like­li­hood of re­flect­ing re­al­ity.

map making

all maps con­tain er­rors, but since er­ror can­not be elimi­nated, the next best thing is to re­duce the mag­ni­tude of our er­rors, and in so do­ing, per­haps also the count. now that we know what the map is and what it is for, we are ready to con­sider the pro­cess of map mak­ing.

the key tech­niques for im­prov­ing your map mak­ing skills over your nat­u­ral abil­ities are well known and stud­ied, but they are also scat­tered and difficult to im­ple­ment. the map mak­ing skills of your an­ces­tors beget your nat­u­ral map­ping skills, but they only had to be good enough map mak­ers to pro­duce you. that still makes them ex­cel­lent map mak­ers, ergo also you, but it does not im­ply you have reached the limits of your map mak­ing abil­ities. sure, you can already make maps that are good enough, but our goal is not a good enough map, but the best map we can make!

since the skills of bet­ter map mak­ing are already well ex­plored by oth­ers, let’s just take a tour (with links) of what you should know so that you can start your jour­ney to bet­ter map mak­ing.

the first thing is to have a good map of the map. this can­not be un­der­es­ti­mated, be­cause your map of the map con­strains your thoughts about the map, and if you want to make bet­ter maps, you need a strong pro­cess for rea­son­ing about the map that al­lows you to im­prove upon it. with­out a good model of the map many map mak­ing tech­niques, com­monly referred to as crit­i­cal think­ing and ra­tio­nal­ity, will seem con­fus­ing. this doesn’t mean you can’t start learn­ing tech­niques be­fore you have your model of the map figured out: it just means you will face con­fu­sion and difficulty as you figure out how to rea­son about the map in a way that makes some of the most pow­er­ful tech­niques use­ful. that’s why i like to write more about the map and less about map mak­ing.

so as­sum­ing you have a good enough map of the map, let’s high­light the core knowl­edge and tech­niques hu­mans have ac­cu­mu­lated to help them make bet­ter maps.

of course this list is in­com­plete. map mak­ing is a skill that re­quires learn­ing a mil­lion lit­tle things and then putting them all into play at once largely with­out ris­ing to the level of con­scious­ness (or forc­ing such a rise when ap­pro­pri­ate). and in truth it is a skill you never stop work­ing on be­cause, if noth­ing else, con­stant vigilance is re­quired to avoid slip­ping back into eas­ier, nat­u­ral meth­ods of map mak­ing. we’ll re­turn fre­quently to the topic of map mak­ing, but for now let this serve as a start on your jour­ney to make bet­ter maps.