Arbital postmortem

Dis­claimer 1: Th­ese views are my own and don’t nec­es­sar­ily re­flect the views of any­one else (Eric, Steph, or Eliezer).

Dis­claimer 2: Most of the events hap­pened at least a year ago. My mem­ory is not par­tic­u­larly great, so the dates are fuzzy and a few things might be slightly out of or­der. But this post has been re­viewed by Eric, Steph, and Eliezer, so it should mostly be okay.

I’m go­ing to list events chronolog­i­cally. At times I’ll in­sert a “Reflec­tion” para­graph, where I’m go­ing to out­line my thoughts as of now. I’ll talk about what I could have done differ­ently and how I would ap­proach a similar prob­lem to­day.

Chap­ter 0: Eliezer pitches Ar­bital and I say ‘no’

Around the sum­mer of 2014 Eliezer ap­proached me with the idea for what later would be­come Ar­bital. At first, I vaguely un­der­stood the idea as some kind of soft­ware to map out knowl­edge. Maybe some­thing like a gi­ant mind map, but not graph­i­cal. I took some time to re­search ex­ist­ing and pre­vi­ous pro­jects in that area and found a huge grave­yard of pro­jects that have been tried. Yes, ba­si­cally all of them were dead. Most were hobby pro­jects, but some seemed pretty se­ri­ous. None were suc­cess­ful, as far as I could tell. I didn’t see how Eliezer’s pro­ject was differ­ent, so I passed on it.

Reflec­tion: To­day, I’d prob­a­bly try to sit down with Eliezer for longer and re­ally try to un­der­stand what he is see­ing that I’m not. It’s likely back then I didn’t have the right skills to ex­tract that in­for­ma­tion, but I think I’m much bet­ter at it to­day.

Reflec­tion: Also, af­ter work­ing with Eliezer for a few years, I’ve got a bet­ter feel­ing for how things he says of­ten seem con­fus­ing /​ out of al­ign­ment /​ tilted, un­til you fi­nally wrap your mind around it, and then it’s crys­tal clear and easy.

Chap­ter 1: Eliezer and I start Arbital

Early Jan­uary 2015 I was sit­ting in my room, tired from look­ing in vain for a de­cent startup idea, when Ar­bital popped back into my mind. There were still a lot of red flags around the idea, but I ra­tio­nal­ized to my­self that given Eliezer’s track record, there was prob­a­bly some­thing good here. And, in the worst case, I’d just cre­ate a tool that would be use­ful to Eliezer alone. That didn’t seem like a bad out­come, so I de­cided to do it. I con­tacted Eliezer, he was still in­ter­ested, and so we started the pro­ject.

Reflec­tion: The de­ci­sion pro­cess sounds a bit silly, but I don’t think it’s a bad one. I re­ally pre­fer to do some­thing de­cently use­ful, rather than sit around wait­ing for some­thing perfect. I also still ap­prove of the heuris­tic of ac­cept­ing quests /​ pro­jects from peo­ple you think are good at com­ing up with quests /​ pro­jects. But if I did it again, I’d definitely put a lot more effort up­front to un­der­stand the en­tire vi­sion be­fore com­mit­ting to it.

Reflec­tion: Paul Gra­ham wrote in one of his es­says that it’s okay (though not ideal) to ini­tially build a product for just one user. There are, of course, sev­eral caveats. The user needs to use the product ex­ten­sively, oth­er­wise you don’t get the nec­es­sary feed­back on all the fea­tures you’re build­ing. And the user needs to be some­what typ­i­cal of other users you hope to at­tract to the plat­form.

Reflec­tion: Un­for­tu­nately, both of these turned out to be false. I’ll elab­o­rate on the fea­ture us­age be­low. But the “typ­i­cal” part prob­a­bly could have been fore­seen. There are only a few peo­ple in the world who write ex­pla­na­tions at the scale and com­plex­ity that Eliezer does. The clos­est cluster is prob­a­bly peo­ple writ­ing col­lege text­books. So, in the be­gin­ning, I didn’t have any sense for who the first 10-100 users were go­ing to be. That would have been fine if I was just build­ing a tool for Eliezer, but since my goal was ex­plic­itly to cre­ate a for-profit con­sumer startup, this was a big mis­take.

Eliezer pro­vided the product vi­sion and de­sign, and I did all the cod­ing. At first, I thought I’d code for a few months and then we would have an MVP that we could show to a few peo­ple to gather more in­ter­est and get some po­ten­tial users. But, as I be­gan to un­der­stand the over­all vi­sion bet­ter my­self, the ship­ping date be­gan drift­ing fur­ther and fur­ther back. At the time this wor­ried me greatly, be­cause I didn’t want to build a thing that no­body else would use. Eliezer’s ar­gu­ment was that we needed to build a product that was the best tool for a par­tic­u­lar work­flow. (I’m the Startup Founder 1 in con­ver­sa­tion 2.) This made sense to me, but I still felt anx­ious that we were fly­ing blind. So around April, I went around and showed what I had to some peo­ple. There wasn’t much to look at, and what was there wasn’t pretty, so it was mostly me ex­plain­ing the idea. The re­cep­tion was luke­warm. Peo­ple said it seemed in­ter­est­ing, but may be not par­tic­u­larly for them. This was a bit dis­cour­ag­ing, but it was also clear that peo­ple weren’t get­ting the full vi­sion.

Reflec­tion: Sigh, this is com­pli­cated. In gen­eral, I agree that if you are show­ing /​ talk­ing about your product to po­ten­tial users and they are not in­ter­ested then ei­ther you’re talk­ing to the wrong peo­ple, your product isn’t use­ful, or you’re pre­sent­ing it wrong. In the case of Ar­bital, though, I think lack of en­thu­si­asm was due to how hard it was to ex­plain the en­tire vi­sion. There were a lot of mov­ing parts, and a lot of what made Ar­bital good even­tu­ally was the full com­bi­na­tion of all those parts.

Reflec­tion: I think the cor­rect thing to do would have been to cre­ate de­tailed UI screens. Then print them and show them to peo­ple (and Eliezer). This prob­a­bly would have taken a month or two, but it would have been worth­while. The rea­son I never got around to it, aside from the ugh-field around do­ing UI mock­ups, was be­cause it always felt like in a month or two we would be done with the MVP.

Reflec­tion: Eliezer re­quested a lot of fea­tures, and most of them had good jus­tifi­ca­tions for why the fi­nal product needed to have them. But, nei­ther of us was very good at pri­ori­tiz­ing. (I wouldn’t say we were bad, but we prob­a­bly could have sped up the de­vel­op­ment by about 25% if we were bet­ter.) It was only around au­tumn when we fi­nally got bet­ter at it.

Reflec­tion: One such fea­ture was a pretty nifty sys­tem for ques­tions and an­swers. Of course, since no­body was us­ing the plat­form, we didn’t re­ally get any ques­tions or an­swers, so it was hard to test that fea­ture, and main­tain­ing it felt pointless. Another fea­ture: a pri­vate do­main, where you could ba­si­cally have your own pri­vate in­stance of Ar­bital at your_sub­do­­

Around sum­mer of 2015, I fi­nally started to get a grasp for the en­tire vi­sion. The grand plan had five ma­jor prob­lems that needed to be solved: Ex­pla­na­tions → De­bate → No­tifi­ca­tions → Rat­ing → Karma. (Done roughly in that or­der, but also in par­allel.)

Ex­pla­na­tions: Ar­bital as a bet­ter Wikipe­dia. (1, 2) Each page would ex­plain a spe­cific con­cept (as op­posed to Wikipe­dia pages that list a bunch of facts); the sys­tem would cre­ate a se­quence of pages for you to read to un­der­stand a topic, where the se­quence would be tai­lored speci­fi­cally to you based on your prefer­ences and what you already know.

De­bate: Ar­bital could of course be used as a blog. We also wanted to sup­port com­ments (both for wiki and blog pages). We also wanted the dis­cus­sion to be of high qual­ity and cen­tral­ized.

No­tifi­ca­tions: Make sure the user is no­tified about var­i­ous events that might in­ter­est them (e.g. a new com­ment in a thread they are sub­scribed to, a new ar­ti­cle to read). Also, if they are a writer, they need to be no­tified of var­i­ous re­lated events as well (e.g. some­one com­mented, some­one pro­posed an edit).

Rat­ing: How will the sys­tem know which pages, ex­pla­na­tions, or com­ments are good? How will the sys­tem be re­sis­tant to peo­ple try­ing to game it to make their pages, ex­pla­na­tions, or com­ments ap­pear bet­ter than they are? If we do this right, we could re­place Yelp (or other ser­vices whose pri­mary func­tion is to provide rat­ings).

Karma: How will we rate users? How will their rat­ings af­fect what they can do? How do rat­ings in­ter­act be­tween do­mains (e.g. math do­main vs. art do­main)?

Later that year Eliezer wrote a 55 page doc­u­ment de­scribing Ar­bital and how and why it was differ­ent and nec­es­sary. (If Eliezer ever gets around to it, he might edit and pub­lish it at some point. I’m mostly men­tion­ing it here to un­der­line the size and com­plex­ity of the pro­ject.)

Reflec­tion: Once I un­der­stood how Ar­bital was differ­ent, it was clear that no pre­vi­ous (nor cur­rent) pro­ject has even come close to try­ing to cap­ture that vi­sion. Over the years I’ve had a lot of peo­ple send me mes­sages that they or their friend were work­ing on a similar pro­ject. And it’s true, for most peo­ple who give a cur­sory glance at Ar­bital, it seems similar to the other “or­ga­nize all knowl­edge” pro­jects. But I’ll still main­tain that Ar­bital is a differ­ent kind of beast. And cer­tainly in scope and am­bi­tion, I haven’t seen any­thing close.

Reflec­tion: Now you can prob­a­bly see how the meme of “Ar­bital will solve that too” was born. It was a hugely am­bi­tious pro­ject for sure, but look­ing back the only prob­lem with that was that for a while we just didn’t have a good, short ex­pla­na­tion of what Ar­bital was. This made it hard to talk to peo­ple about the pro­ject and get them ex­cited. It also made pri­ori­tiz­ing fea­tures more difficult.

So, the first ma­jor prob­lem we wanted to solve was Ex­pla­na­tions. If we solved it well, it’s pos­si­ble we could be­come the next Wikipe­dia (or at least a much bet­ter Quora). Our goal was for Ar­bital to be the best tool to write and or­ga­nize on­line ex­pla­na­tions. The pri­mary topic we wanted to ex­plain was, of course, AI safety. But we rea­soned that if we just had AI safety con­tent, es­pe­cially if it was mostly writ­ten by Eliezer, the web­site wouldn’t be­come gen­er­ally used and its con­tent widely ac­cepted. (And then we definitely wouldn’t be­come the next Wikipe­dia.) This is why later we fo­cused mostly on math ex­pla­na­tions.

At the end of 2015 we launched a pri­vate beta ver­sion for MIRI. A few weeks be­fore, I sat down with a UX de­signer, Greg Schwartz. We spent a few ses­sions go­ing over all the screens and re­design­ing them to be sim­pler and more un­der­stand­able. He of­ten pushed me to sim­plify the pro­ject and drop var­i­ous fea­tures. I also had an­other friend look at UI and help with font and col­ors. This was definitely time well spent (only about a month), and we later got many com­pli­ments on the look and feel of the web­site.

Reflec­tion: It oc­curs to me now that while Greg’s feed­back had some speci­fics wrong, it was over­all cor­rect in that it was point­ing out a deep prob­lem: the pro­ject had too many mov­ing parts and a lot of those parts weren’t re­ally used. It would have been hard to guess which parts would end up nec­es­sary, but the right solu­tion was to find more users who would want to use the plat­form now (or very soon) and talk to them.

I was ex­cited about the launch, be­cause I thought that fi­nally some peo­ple aside from Eliezer would be us­ing Ar­bital. Un­for­tu­nately, it was only many many months later that other peo­ple from MIRI slowly started us­ing it.

Reflec­tion: I think af­ter we reached our “MVP”, I should have switched into “find users” mode. (Ideally, I would have had users lined up at the out­set, but even this timing would have been okay.) For ex­am­ple, I could have pushed for Agent Foun­da­tions fo­rum to be ported to Ar­bital. Even though that was more of a Dis­cus­sion pro­ject, these were very reach­able users, still within the over­all strat­egy. I think we should have used a greedy user ac­qui­si­tion strat­egy, in­stead of try­ing to stick to our rigid se­quen­tial plan.

Reflec­tion: I’d de­scribe one of the main strug­gles of 2015 as: “we need to build a small MVP quickly and get feed­back from users” (Alexei) vs. “users don’t know what they want, and they won’t be able to give you mean­ingful feed­back un­til they see and use the product” (Eliezer). Like I men­tioned above, I think the cor­rect solu­tion here are de­tailed mock­ups.

Reflec­tion: Another strug­gle was: “we need users to make sure we are build­ing things cor­rectly” (Alexei) vs. “I can tell when we are build­ing things cor­rectly, I can get us users as soon as the product is ready” (Eliezer). Un­for­tu­nately, we never got the product “ready” enough to test Eliezer’s claim. I think it would have a taken a long while to get there. But, given how things ended up, it’s pos­si­ble that would have been a bet­ter path.

Chap­ter 2: Eric and Steph join Ar­bital, and we take des­tiny into our own hands

Around April of 2016 Eric Rogstad and Stephanie Zo­lay­var joined the team. We con­tinued fol­low­ing Eliezer’s vi­sion and have him dic­tate fea­tures and their de­sign. Since fo­cus­ing on AI al­ign­ment alone wouldn’t have re­sulted in a re­spected plat­form, we shifted our pri­mary topic to math, speci­fi­cally: in­tu­itive math ex­pla­na­tions.

Reflec­tion: When we ran this idea by peo­ple, we got a lot of pos­i­tive feed­back. A lot of peo­ple said they wanted that kind of web­site, but it took me some time to re­al­ize that ev­ery­one wanted to read in­tu­itive math ex­pla­na­tions, but al­most no­body would ac­tu­ally spend the time cre­at­ing them, even if they could in prin­ci­ple.

We in­vited some peo­ple to write the con­tent. We hosted a writ­ing party. We had a Slack chan­nel, where with Eric Bruy­lant’s help we built a small com­mu­nity. Some peo­ple wrote pretty good math ex­pla­na­tions, but over­all things moved way too slow. We talked to some of our users; we tried var­i­ous things, like cre­at­ing pro­jects. But, we sim­ply didn’t have enough writ­ers, and we didn’t know how to find more.

Reflec­tion: I think we should have dropped most of the de­vel­op­ment and fo­cused on user ac­qui­si­tion at this point. There were sev­eral times when I con­sid­ered pivot­ing to a “math blog­ging” plat­form, but it felt like too big of a shift from wiki-fo­cused plan we were pur­su­ing. Again, I think a greedy “ac­quire users now!” strat­egy would have served us well.

One of the biggest fea­tures we built around this time was dy­namic ex­pla­na­tions. A lot of effort went into de­sign­ing and im­ple­ment­ing a sys­tem of req­ui­sites. Ba­si­cally each page could teach and/​or re­quire cer­tain req­ui­sites, which were other pages. It was not clear what over­all on­tol­ogy we wanted, so it took us a while to iter­ate this fea­ture and we ended up with a lot of edge cases. We built some­thing that worked okay, but, again, it was hard to test be­cause there wasn’t quite enough dense con­tent.

Reflec­tion: I’d say we iter­ated that fea­ture for way too long. In part this was be­cause Eliezer was con­sis­tently not satis­fied with what we im­ple­mented. At some point things be­came way too “hacky.” I think if we sim­ply had more pages and more peo­ple con­struct­ing ex­pla­na­tions, it would have helped us an­swer a lot of the in­ter­nal de­bates we had. But in­stead we were try­ing to wran­gle a set of about 30 pages to work in just the right way. We should have left the fea­ture as good enough and moved on. (But re­ally, we should have been get­ting more users.)

Not only was it hard to find writ­ers, but the ex­pla­na­tions were hard to write as well. In gen­eral, writ­ing mod­u­lar ex­pla­na­tions is very hard. Dou­bly so, when you also want to string those ex­pla­na­tions to­gether to form a co­her­ent se­quence.

Reflec­tion: we were also try­ing to build a two-sided mar­ket­place. We needed writ­ers, but writ­ers wanted read­ers, but read­ers wanted good con­tent. I think the cor­rect way to solve that would have been to at­tract peo­ple with ex­ist­ing blogs /​ read­er­ship to switch to Ar­bital and bring their au­di­ence with them.

Reflec­tion: Team-wise we ab­solutely needed some­one who would be go­ing af­ter users all the time and talk­ing to them, re­cruit­ing them, mar­ket­ing, etc… No­body on the team had ex­pe­rience with or af­finity for do­ing that.

To help us show­case the plat­form, Eliezer wrote the Bayes’ Rule Guide. We’ve went through sev­eral iter­a­tions of it over the course of a few months, tweak­ing fea­tures and im­prov­ing re­ten­tion met­rics. The some­what dense set of pages helped us test a few fea­tures eas­ier. Lots of peo­ple read the guide and loved it, but it wasn’t ob­vi­ous if Ar­bital for­mat helped vs. Eliezer’s writ­ing was good. I think peo­ple also didn’t ap­pre­ci­ate the magic that hap­pened be­hind the scene. (How do you com­mu­ni­cate to a reader that they could have had a much worse read­ing ex­pe­rience but didn’t?)

Nate Soares helped us by writ­ing the Log­a­r­ithm Guide. We thought if we could pro­duce good se­quences like that fre­quently and post them on­line, we might slowly get trac­tion. Un­for­tu­nately, it’s re­ally time con­sum­ing to pro­duce con­tent that good, and there are just sim­ply not that many peo­ple who can write con­tent of that qual­ity (and have the time to do it for free).

Here is what the front page looked like around that time. At the height of it, we had about a dozen reg­u­lar users who would come and write a few pages ev­ery week. They en­joyed the small com­mu­nity we had and fre­quently hung out in our Slack chan­nel. They wanted to write math ex­pla­na­tions for them­selves and their friends. I don’t re­mem­ber how many read­ers we had, but it was around 50-200 /​ day, most of them redi­rected from Eliezer’s old guide to Bayes’ The­o­rem.

In Au­gust we raised a $300k pre-seed round. We had about 9 in­vestors. Most of them in­vested be­cause of Eliezer, but a few knew me per­son­ally as well.

Also around that time, it be­came clear to us that things just weren’t go­ing well. The pri­mary is­sue was that we com­pletely re­lied on Eliezer to provide guidance to which fea­tures to im­ple­ment and how to im­ple­ment them. Fre­quently when we tried to do things our way, we were over­ruled. (Never with­out a de­cent rea­son, but I think in many of those cases ei­ther side had merit. Go­ing with Eliezer’s point of view meant we fre­quently ended up blocked on him, be­cause we couldn’t pre­dict the next steps.) Also, since Eliezer was the only per­son se­ri­ously us­ing the product, there wasn’t enough rapid feed­back for many of the fea­tures. And since we wasn’t in the office with us ev­ery day, we were of­ten blocked.

So, we de­cided to take the mat­ter into our own hands. The three of us would de­cide what to do, and we would oc­ca­sion­ally talk to Eliezer to get his in­put on spe­cific things.

Reflec­tion: Work­ing with Eliezer was in­ter­est­ing to say the least. He cer­tainly had a great over­all vi­sion for the product; one that I’m still as­ton­ished by to this day. He of­ten had good in­sight into spe­cific fea­tures and how to im­ple­ment them. But some­times he would get way too bogged down by cer­tain de­tails and spend longer on a fea­ture than I thought was nec­es­sary. (In most of those cases he need things to work a cer­tain way to solve a par­tic­u­lar prob­lem he had, but it was wast­ing our time be­cause we were build­ing some­thing ul­tra spe­cific to his use case.) This was es­pe­cially painful for fea­tures that no­body, in­clud­ing Eliezer, would end up us­ing.

Reflec­tion: Eliezer also had a ten­dency some­times to over­com­pli­cate things and de­signs sys­tems that I could barely wrap my head around. (I of­ten joked that we would end up build­ing a web­site that only one per­son in the world could use.) But then again, there were also many mo­ments where a com­pli­cated, messy fea­ture would sud­denly click into place, and then it seemed ob­vi­ous and sim­ple.

Reflec­tion: I’m tempted to draw a les­son like: never ever build a product you don’t un­der­stand your­self. But if I did that, I’d cer­tainly miss a huge learn­ing op­por­tu­nity of work­ing with Eliezer and lev­el­ing up my product and UX skills. So, in­stead, I think the les­son is: if you’re run­ning the pro­ject, never ever do any­thing that doesn’t make sense to you. As soon as you start del­e­gat­ing /​ ac­cept­ing things that don’t make sense, you muddy the wa­ter for your­self. Now the strat­egy has opaque com­po­nents that you don’t un­der­stand, can’t ex­plain, and some­times ac­tively dis­agree with. There is just no way you can move at the nec­es­sary startup speed like that, and you’re also not learn­ing from your mis­takes.

Reflec­tion: This is es­pe­cially true with re­spect to the over­all strat­egy. Yes, maybe some paths are ob­jec­tively bet­ter or eas­ier. But if it’s not one that makes sense to you, if it’s not one you can ex­e­cute, then you should take an­other path.

Reflec­tion: Look­ing at ar­ to­day, I’m ac­tu­ally still very much im­pressed with it. It’s a good piece of soft­ware, and if I wanted to write ex­pla­na­tions, I think I’d be hard pressed to find a bet­ter web­site. Iron­i­cally, the large part of what makes it re­ally good are all the fea­tures that it has.

Chap­ter 3: Pivot to discussion

It was clear that we couldn’t scale a com­mu­nity around math. So we de­cided to pivot. It wasn’t a clean and easy pivot; if I re­mem­ber cor­rectly it took us about a month of strug­gling and de­liber­at­ing to de­cide that our cur­rent ap­proach wasn’t work­ing and then set­tle on a new one.

We de­cided to skip the Ex­pla­na­tions part and go straight for the Dis­cus­sion. We started build a new de­sign around claims. A claim is a page with a propo­si­tion that users can vote on by as­sign­ing a prob­a­bil­ity es­ti­mate or by mark­ing the de­gree of (dis)agree­ment. The idea was that peo­ple would blog on Ar­bital, and cre­ate pages for claims they dis­cussed. Peo­ple could vote on claims, and thus ev­ery­one could see where peo­ple mostly agreed and mostly dis­agreed. Claims could also be reused in other blog posts by the same au­thor or other peo­ple.

We kept most of the origi­nal ar­chi­tec­ture, but re­made the home­page. We also shifted the fo­cus to the re­gional ra­tio­nal­ist com­mu­nity. We did mul­ti­ple user in­ter­views. We did UI mock­ups. We talked to some ra­tio­nal­ist blog­gers and got some mild sup­port.

One of my fa­vorite ar­ti­facts to come out from that time pe­riod is this SlateS­tarCodex pre­dic­tions page.

Reflec­tion: At the time, I think the pivot de­ci­sion was cor­rect. And if we con­tinued go­ing with it, it’s pos­si­ble Ar­bital would have be­come LW 2.0, though that wasn’t ex­actly our in­ten­tion at the time.

Reflec­tion: One thing we messed up dur­ing this time was dilut­ing lead­er­ship. Since Eliezer was no longer in charge of product, the re­spon­si­bil­ity fell on all of us. This re­sulted in many many dis­cus­sion about what to build, how to build it, down the minute de­tails. Our pace re­ally slowed down and it took us a while to patch it up.

Chap­ter 4: End of Ar­bital 1.0

In the be­gin­ning of 2017 I ex­pe­rienced my first burnout. There was sim­ply no way I could work, so I apol­o­gized to the team and spent a month play­ing video games, which I des­per­ately needed. This gave me the time, space, and dis­tance to think about the pro­ject. When I came back, I sat down with the team and we had an ex­ten­sive dis­cus­sion about the di­rec­tion of the com­pany.

Eric and Steph wanted to stay the course. I no longer be­lieved that was go­ing to work, and I wanted to try a differ­ent ap­proach. My biggest re­al­iza­tion dur­ing my break was that peo­ple (and in this case speci­fi­cally: most ra­tio­nal­ists) were not ac­tu­ally in­ter­ested in putting any se­ri­ous effort in im­prov­ing the state of on­line de­bate. While al­most ev­ery­one wanted bet­ter on­line dis­cus­sions, just like with math ex­pla­na­tions, al­most no­body was will­ing to put in any kind of work.

Fur­ther­more, when we talked to a few ra­tio­nal­ists di­rectly, I just didn’t get the feel­ing of gen­uine helpful­ness or en­thu­si­asm. This was up­set­ting, be­cause there aren’t that many big pro­jects that the com­mu­nity does. So when I was do­ing Ar­bital, I guess I ex­pected that more peo­ple would be on board, that more peo­ple would put in a bit of an ex­tra effort to help us. But at best peo­ple put in min­i­mal work (to satisfy us or them­selves, I’m not sure). How­ever, there was a limit to how up­set I could be, be­cause I very clearly rec­og­nized the same trait in me. So, while it’s still a sad state of af­fairs, I’d be a hyp­ocrite for be­ing up­set with any par­tic­u­lar per­son.

Reflec­tion: I think the ra­tio­nal­ity com­mu­nity can pro­duce great in­sights, but mostly due to in­di­vi­d­ual effort. There are great posts, but they rarely lead to pro­longed con­ver­sa­tions. And you very rarely see de­bates sum­ma­rized for pub­lic con­sump­tion. (No won­der, it takes a lot of time and hard work!) There are a few coun­terex­am­ples, but I think they prove the point by how much they stand out. (Best re­cent ex­am­ple I can think of is Jes­sica Tay­lor me­di­at­ing a dis­cus­sion be­tween Eliezer and Paul Chris­ti­ano and then writ­ing it up.) (And, of course, not only do those things need to be writ­ten, but they also have to be read! And who has time to read…)

Reflec­tion: I’m pleas­antly sur­prised by the cur­rently ac­tive LW 2.0. I think this is some ev­i­dence against my claim, but over­all, I still think that when it comes to build­ing out more de­tailed mod­els with ex­plicit claims, es­pe­cially when it in­volves work­ing with other peo­ple, most peo­ple are not will­ing to put in the ex­tra work. (Espe­cially if their name isn’t at­tached to it.)

It was clear to me how to ad­dress this is­sue. Peo­ple are will­ing to do what they are already do­ing. In par­tic­u­lar: blog­ging. It didn’t seem that hard to take the soft­ware we had and re­ally op­ti­mize it to be a bet­ter blog­ging plat­form, at least for some au­di­ence (like math blog­gers). And it seemed ob­vi­ous to me that we would at least get some users that way. The key differ­ence from our path at the time was that in­stead of solv­ing the Dis­cus­sion prob­lem and try­ing to get peo­ple to do new things, we’d sim­ply fo­cus on build­ing a bet­ter tool for a thing peo­ple already do. Then once we had peo­ple on our plat­form, we could help im­prove the on­go­ing dis­cus­sions.

Reflec­tion: This was me fi­nally chan­nel­ing the “greedy” user ac­qui­si­tion strat­egy.

At some point dur­ing the de­bate we con­sid­ered try­ing both pro­jects in par­allel, but at the end, Eric and Steph de­cided to leave. I’d take Ar­bital in the new di­rec­tion by my­self. (Huge thank you to Anna Sal­mon for helping to me­di­ate that dis­cus­sion. I’d say it went pretty well, all things con­sid­ered.)

Chap­ter 5: Ar­bital 2.0

I spent the rest of 2017 work­ing on Ar­bital 2.0. At first it was go­ing very well. The vi­sion felt very clear to me, I had my mind to­tally wrapped around it, and all parts of the strat­egy made sense. But for some rea­son, around sum­mer of 2017 it be­came re­ally hard to work. I spent a lot of the time try­ing to code, but be­ing un­able to. Even though in­tel­lec­tu­ally I be­lieved in the idea very much, my spirit was burned out /​ my Sys­tem 1 just didn’t be­lieve in the pro­ject. After strug­gling with it on and off for the re­main­ing half of the year, I fi­nally had to ad­mit to my­self that it just didn’t have enough mo­men­tum to suc­ceed.

(The rest of this chap­ter is a Reflec­tion.)

My best guess is that I was burnt out again. Even though I didn’t feel as bad as I did in Jan­uary, the feel­ing of be­ing un­able to even touch the lap­top was very similar.

For those cu­ri­ous and for those look­ing for a startup idea, I’m go­ing to de­scribe my plan for Ar­bital 2.0. In short, it’s Tum­blr for math­e­mat­i­ci­ans. You could use it as a blog, but it’s re­ally a so­cial net­work. What makes it rad­i­cally differ­ent is the abil­ity for one per­son to cre­ate and own mul­ti­ple topic-cen­tered chan­nels. (One big is­sue I see with FB is that it doesn’t scale well with the num­ber of friends. With most friends I only want to talk about cer­tain spe­cific top­ics. But FB is broad­cast-to-all by de­sign.) On Ar­bital 2.0, I would be able to post about im­prov, Rick and Morty, AI, scuba-div­ing, and all my other in­ter­ests to differ­ent chan­nels. Peo­ple could sub­scribe to the chan­nels they were in­ter­ested in. So if you never wanted to listen to poli­tics, you wouldn’t fol­low peo­ple on their poli­ti­cal chan­nel. (Or hide all posts with #poli­tics.) Each chan­nel could be grown into a com­mu­nity, where other peo­ple could sub­mit their posts too.

I still think this ap­proach is very likely to work:

  • Write the soft­ware. (I have a 70% baked ver­sion.)

  • Go to math­e­mat­i­ci­ans and offer to host their blog on Ar­bital. (Why math­e­mat­i­ci­ans? There is ba­si­cally no good blog­ging soft­ware with great sup­port for LaTeX. That fea­ture plus a few oth­ers will con­vince many peo­ple to switch or at least to try Ar­bital.) When I cold emailed 100+ math blog­gers, I got a good num­ber of pretty en­thu­si­as­tic re­sponses. Path for 0 to 1000 users seems straight­for­ward.

  • Most math blog­gers also blog about other things. This nat­u­rally will lead them to use the chan­nels fea­ture.

  • Many peo­ple have top­ics that they can’t dis­cuss on Face­book be­cause they don’t want to spam their friends. (I’d make at least one Rick and Morty post a day if I could.) Ar­bital would be the perfect out­let for those. (I’d as­sign 20% prob­a­bil­ity that it’s pos­si­ble to skip and suc­ceed with this step di­rectly with­out both­er­ing with re­cruit­ing math blog­gers first.)

  • Then fol­low the origi­nal Ar­bital plan: So­cial Net­work → Ex­pla­na­tions → De­bate → No­tifi­ca­tions → Rat­ing → Karma.

  • And, of course, put the en­tire thing on a blockchain to make crazy money in the mean­time. ;)

It’s pretty clear to me that for Ar­bital to work at scale it has to be a so­cial net­work. Part of why I don’t think most other paths will work is that so­cial me­dia ate all the free time. It’s not that peo­ple be­came lazy, it’s just when it’s a choice be­tween spend­ing an­other 15 mins on FB or spend­ing that 15 mins cre­at­ing and link­ing claims to your new blog post, most peo­ple will choose FB. (And while FB is the best ex­am­ple, the prob­lem is more wide­spread, of course. Every­thing be­came more ad­dic­tive.) This is why the new ap­proach was to cre­ate a so­cial net­work that’s bet­ter than FB and would al­low you to man­age your time and at­ten­tion. And then from there we could ac­tu­ally put that saved time and at­ten­tion to more use­ful things. (Although, I’m still scep­ti­cal that there are enough peo­ple who will have con­struc­tive de­bates to war­rant all this effort.)

One rea­son I’m not pur­su­ing this right now (aside from be­ing burnt out with the whole en­ter­prise) is that it no longer ob­vi­ously helps with AI safety. If you re­call, one of the as­sump­tions was that if we did Ar­bital speci­fi­cally for AI safety, the web­site wouldn’t get enough cred­i­bil­ity. With some re­cent de­vel­op­ments, I think that’s no longer the case. AI safety in gen­eral is more ac­cepted, as well as MIRI’s re­search in par­tic­u­lar. So, if I did any work in this space again, I’d, iron­i­cally enough, go back to the origi­nal vi­sion of cre­at­ing an ex­pla­na­tion web­site for Eliezer and other peo­ple who wanted to write about AI safety. (But, ac­tu­ally, I think the bot­tle­neck is good con­tent and peo­ple ac­tu­ally read­ing it.)


What’s go­ing to hap­pen with Ar­bital?

I’m cur­rently in the pro­cess of shut­ting down the com­pany. All the soft­ware and IP is go­ing to be turned over to MIRI. A few peo­ple ex­pressed in­ter­est in hav­ing Ar­bital be open sourced, in­clud­ing one of our in­vestors, so that’s likely to hap­pen.

Ar­bital tech stack

Ar­bital 1.0: Golang on BE, An­gu­lar 1.0 on FE, MySQL DB, mark­down ed­i­tor.
Ar­bital 2.0: NodeJS on BE, Re­act on FE, Aran­goDB, SlateJS ed­i­tor. (Much much bet­ter choices all around!)

How much do you think tech­ni­cal skill mat­tered on the mar­gin?

A lot. This was a pretty com­plex pro­ject, so man­ag­ing code com­plex­ity was im­por­tant. We also needed to con­tin­u­ously op­ti­mize things to make sure ev­ery­thing loaded de­cently fast. We had to make sure there weren’t many bugs, be­cause most things were user-fac­ing. And be­ing able to code de­cently fast helped a lot, since the amount of fea­tures we had to im­ple­ment was fairly large.


I hated work­ing with our lawyers. This was may be the most frus­trat­ing part of the en­tire pro­ject.

Les­son 1: work only with a team who is recom­mended to you per­son­ally by some­one you trust and who has worked with them be­fore.

Les­son 2: Ask how much some­thing will take up­front. If the lawyer wants to spend more than an hour on any­thing, have them dou­ble check that you want it done. Have them send you a weekly re­port of time spent.

Les­son 3: Con­sider just not go­ing with lawyers and us­ing stan­dard pa­per­work. Be­fore Series A it just doesn’t mat­ter and you can re­struc­ture things any way you want later.

Sin­gle founder

Be­ing a sin­gle founder is not great, but there are ac­tual rea­sons for it that in prin­ci­ple could be miti­gated.

It’s un­likely you have all the skills. (Note that the situ­a­tion is not that differ­ent if you have a co-founder, but they are very similar to you.) More im­por­tant than the skills though, is your per­son­al­ity /​ in­cli­na­tion. Per­son­ally, I’d rather code than talk to users. So an ideal co-founder for me would be some­one out­go­ing, who’d pre­fer talk­ing to users than do­ing other things.

Not hav­ing some­one to talk to day-to-day means you might end up with a tun­nel vi­sion /​ stuck /​ do­ing unim­por­tant things /​ for­get­ting to take a step back /​ mak­ing ba­sic mis­takes. Hav­ing some­one to talk to on fre­quent ba­sis is im­por­tant.

It feels bad when you don’t work /​ are stuck and the pro­ject doesn’t move for­ward. When you’re work­ing with some­one else, they usu­ally make progress even when you don’t.


It’s im­por­tant that in each area there is a sin­gle per­son who is ul­ti­mately re­spon­si­ble for it. Product: one per­son. Busi­ness: one per­son. And, over­all, for the com­pany: one per­son. As­sign­ing a re­spon­si­bil­ity to more than one per­son will sig­nifi­cantly in­crease the com­mu­ni­ca­tion over­head and slow things down.

Again, a heart­felt thank you to ev­ery­one who has par­ti­ci­pated in this ad­ven­ture. It has been quite a jour­ney and, at the end of the day, I wouldn’t take any of it back.