Arbital postmortem

[alexei]

Disclaimer 1: These views are my own and don’t necessarily reflect the views of anyone else (Eric, Steph, or Eliezer).

Disclaimer 2: Most of the events happened at least a year ago. My memory is not particularly great, so the dates are fuzzy and a few things might be slightly out of order. But this post has been reviewed by Eric, Steph, and Eliezer, so it should mostly be okay.

I’m going to list events chronologically. At times I’ll insert a “Reflection” paragraph, where I’m going to outline my thoughts as of now. I’ll talk about what I could have done differently and how I would approach a similar problem today.

Chapter 0: Eliezer pitches Arbital and I say ‘no’

Around the summer of 2014 Eliezer approached me with the idea for what later would become Arbital. At first, I vaguely understood the idea as some kind of software to map out knowledge. Maybe something like a giant mind map, but not graphical. I took some time to research existing and previous projects in that area and found a huge graveyard of projects that have been tried. Yes, basically all of them were dead. Most were hobby projects, but some seemed pretty serious. None were successful, as far as I could tell. I didn’t see how Eliezer’s project was different, so I passed on it.

Reflection: Today, I’d probably try to sit down with Eliezer for longer and really try to understand what he is seeing that I’m not. It’s likely back then I didn’t have the right skills to extract that information, but I think I’m much better at it today.

Reflection: Also, after working with Eliezer for a few years, I’ve got a better feeling for how things he says often seem confusing /​ out of alignment /​ tilted, until you finally wrap your mind around it, and then it’s crystal clear and easy.

Chapter 1: Eliezer and I start Arbital

Early January 2015 I was sitting in my room, tired from looking in vain for a decent startup idea, when Arbital popped back into my mind. There were still a lot of red flags around the idea, but I rationalized to myself that given Eliezer’s track record, there was probably something good here. And, in the worst case, I’d just create a tool that would be useful to Eliezer alone. That didn’t seem like a bad outcome, so I decided to do it. I contacted Eliezer, he was still interested, and so we started the project.

Reflection: The decision process sounds a bit silly, but I don’t think it’s a bad one. I really prefer to do something decently useful, rather than sit around waiting for something perfect. I also still approve of the heuristic of accepting quests /​ projects from people you think are good at coming up with quests /​ projects. But if I did it again, I’d definitely put a lot more effort upfront to understand the entire vision before committing to it.

Reflection: Paul Graham wrote in one of his essays that it’s okay (though not ideal) to initially build a product for just one user. There are, of course, several caveats. The user needs to use the product extensively, otherwise you don’t get the necessary feedback on all the features you’re building. And the user needs to be somewhat typical of other users you hope to attract to the platform.

Reflection: Unfortunately, both of these turned out to be false. I’ll elaborate on the feature usage below. But the “typical” part probably could have been foreseen. There are only a few people in the world who write explanations at the scale and complexity that Eliezer does. The closest cluster is probably people writing college textbooks. So, in the beginning, I didn’t have any sense for who the first 10-100 users were going to be. That would have been fine if I was just building a tool for Eliezer, but since my goal was explicitly to create a for-profit consumer startup, this was a big mistake.

Eliezer provided the product vision and design, and I did all the coding. At first, I thought I’d code for a few months and then we would have an MVP that we could show to a few people to gather more interest and get some potential users. But, as I began to understand the overall vision better myself, the shipping date began drifting further and further back. At the time this worried me greatly, because I didn’t want to build a thing that nobody else would use. Eliezer’s argument was that we needed to build a product that was the best tool for a particular workflow. (I’m the Startup Founder 1 in conversation 2.) This made sense to me, but I still felt anxious that we were flying blind. So around April, I went around and showed what I had to some people. There wasn’t much to look at, and what was there wasn’t pretty, so it was mostly me explaining the idea. The reception was lukewarm. People said it seemed interesting, but may be not particularly for them. This was a bit discouraging, but it was also clear that people weren’t getting the full vision.

Reflection: Sigh, this is complicated. In general, I agree that if you are showing /​ talking about your product to potential users and they are not interested then either you’re talking to the wrong people, your product isn’t useful, or you’re presenting it wrong. In the case of Arbital, though, I think lack of enthusiasm was due to how hard it was to explain the entire vision. There were a lot of moving parts, and a lot of what made Arbital good eventually was the full combination of all those parts.

Reflection: I think the correct thing to do would have been to create detailed UI screens. Then print them and show them to people (and Eliezer). This probably would have taken a month or two, but it would have been worthwhile. The reason I never got around to it, aside from the ugh-field around doing UI mockups, was because it always felt like in a month or two we would be done with the MVP.

Reflection: Eliezer requested a lot of features, and most of them had good justifications for why the final product needed to have them. But, neither of us was very good at prioritizing. (I wouldn’t say we were bad, but we probably could have sped up the development by about 25% if we were better.) It was only around autumn when we finally got better at it.

Reflection: One such feature was a pretty nifty system for questions and answers. Of course, since nobody was using the platform, we didn’t really get any questions or answers, so it was hard to test that feature, and maintaining it felt pointless. Another feature: a private domain, where you could basically have your own private instance of Arbital at your_subdomain.arbital.com.

Around summer of 2015, I finally started to get a grasp for the entire vision. The grand plan had five major problems that needed to be solved: Explanations → Debate → Notifications → Rating → Karma. (Done roughly in that order, but also in parallel.)

Explanations: Arbital as a better Wikipedia. (1, 2) Each page would explain a specific concept (as opposed to Wikipedia pages that list a bunch of facts); the system would create a sequence of pages for you to read to understand a topic, where the sequence would be tailored specifically to you based on your preferences and what you already know.

Debate: Arbital could of course be used as a blog. We also wanted to support comments (both for wiki and blog pages). We also wanted the discussion to be of high quality and centralized.

Notifications: Make sure the user is notified about various events that might interest them (e.g. a new comment in a thread they are subscribed to, a new article to read). Also, if they are a writer, they need to be notified of various related events as well (e.g. someone commented, someone proposed an edit).

Rating: How will the system know which pages, explanations, or comments are good? How will the system be resistant to people trying to game it to make their pages, explanations, or comments appear better than they are? If we do this right, we could replace Yelp (or other services whose primary function is to provide ratings).

Karma: How will we rate users? How will their ratings affect what they can do? How do ratings interact between domains (e.g. math domain vs. art domain)?

Later that year Eliezer wrote a 55 page document describing Arbital and how and why it was different and necessary. (If Eliezer ever gets around to it, he might edit and publish it at some point. I’m mostly mentioning it here to underline the size and complexity of the project.)

Reflection: Once I understood how Arbital was different, it was clear that no previous (nor current) project has even come close to trying to capture that vision. Over the years I’ve had a lot of people send me messages that they or their friend were working on a similar project. And it’s true, for most people who give a cursory glance at Arbital, it seems similar to the other “organize all knowledge” projects. But I’ll still maintain that Arbital is a different kind of beast. And certainly in scope and ambition, I haven’t seen anything close.

Reflection: Now you can probably see how the meme of “Arbital will solve that too” was born. It was a hugely ambitious project for sure, but looking back the only problem with that was that for a while we just didn’t have a good, short explanation of what Arbital was. This made it hard to talk to people about the project and get them excited. It also made prioritizing features more difficult.

So, the first major problem we wanted to solve was Explanations. If we solved it well, it’s possible we could become the next Wikipedia (or at least a much better Quora). Our goal was for Arbital to be the best tool to write and organize online explanations. The primary topic we wanted to explain was, of course, AI safety. But we reasoned that if we just had AI safety content, especially if it was mostly written by Eliezer, the website wouldn’t become generally used and its content widely accepted. (And then we definitely wouldn’t become the next Wikipedia.) This is why later we focused mostly on math explanations.

At the end of 2015 we launched a private beta version for MIRI. A few weeks before, I sat down with a UX designer, Greg Schwartz. We spent a few sessions going over all the screens and redesigning them to be simpler and more understandable. He often pushed me to simplify the project and drop various features. I also had another friend look at UI and help with font and colors. This was definitely time well spent (only about a month), and we later got many compliments on the look and feel of the website.

Reflection: It occurs to me now that while Greg’s feedback had some specifics wrong, it was overall correct in that it was pointing out a deep problem: the project had too many moving parts and a lot of those parts weren’t really used. It would have been hard to guess which parts would end up necessary, but the right solution was to find more users who would want to use the platform now (or very soon) and talk to them.

I was excited about the launch, because I thought that finally some people aside from Eliezer would be using Arbital. Unfortunately, it was only many many months later that other people from MIRI slowly started using it.

Reflection: I think after we reached our “MVP”, I should have switched into “find users” mode. (Ideally, I would have had users lined up at the outset, but even this timing would have been okay.) For example, I could have pushed for Agent Foundations forum to be ported to Arbital. Even though that was more of a Discussion project, these were very reachable users, still within the overall strategy. I think we should have used a greedy user acquisition strategy, instead of trying to stick to our rigid sequential plan.

Reflection: I’d describe one of the main struggles of 2015 as: “we need to build a small MVP quickly and get feedback from users” (Alexei) vs. “users don’t know what they want, and they won’t be able to give you meaningful feedback until they see and use the product” (Eliezer). Like I mentioned above, I think the correct solution here are detailed mockups.

Reflection: Another struggle was: “we need users to make sure we are building things correctly” (Alexei) vs. “I can tell when we are building things correctly, I can get us users as soon as the product is ready” (Eliezer). Unfortunately, we never got the product “ready” enough to test Eliezer’s claim. I think it would have a taken a long while to get there. But, given how things ended up, it’s possible that would have been a better path.

Chapter 2: Eric and Steph join Arbital, and we take destiny into our own hands

Around April of 2016 Eric Rogstad and Stephanie Zolayvar joined the team. We continued following Eliezer’s vision and have him dictate features and their design. Since focusing on AI alignment alone wouldn’t have resulted in a respected platform, we shifted our primary topic to math, specifically: intuitive math explanations.

Reflection: When we ran this idea by people, we got a lot of positive feedback. A lot of people said they wanted that kind of website, but it took me some time to realize that everyone wanted to read intuitive math explanations, but almost nobody would actually spend the time creating them, even if they could in principle.

We invited some people to write the content. We hosted a writing party. We had a Slack channel, where with Eric Bruylant’s help we built a small community. Some people wrote pretty good math explanations, but overall things moved way too slow. We talked to some of our users; we tried various things, like creating projects. But, we simply didn’t have enough writers, and we didn’t know how to find more.

Reflection: I think we should have dropped most of the development and focused on user acquisition at this point. There were several times when I considered pivoting to a “math blogging” platform, but it felt like too big of a shift from wiki-focused plan we were pursuing. Again, I think a greedy “acquire users now!” strategy would have served us well.

One of the biggest features we built around this time was dynamic explanations. A lot of effort went into designing and implementing a system of requisites. Basically each page could teach and/​or require certain requisites, which were other pages. It was not clear what overall ontology we wanted, so it took us a while to iterate this feature and we ended up with a lot of edge cases. We built something that worked okay, but, again, it was hard to test because there wasn’t quite enough dense content.

Reflection: I’d say we iterated that feature for way too long. In part this was because Eliezer was consistently not satisfied with what we implemented. At some point things became way too “hacky.” I think if we simply had more pages and more people constructing explanations, it would have helped us answer a lot of the internal debates we had. But instead we were trying to wrangle a set of about 30 pages to work in just the right way. We should have left the feature as good enough and moved on. (But really, we should have been getting more users.)

Not only was it hard to find writers, but the explanations were hard to write as well. In general, writing modular explanations is very hard. Doubly so, when you also want to string those explanations together to form a coherent sequence.

Reflection: we were also trying to build a two-sided marketplace. We needed writers, but writers wanted readers, but readers wanted good content. I think the correct way to solve that would have been to attract people with existing blogs /​ readership to switch to Arbital and bring their audience with them.

Reflection: Team-wise we absolutely needed someone who would be going after users all the time and talking to them, recruiting them, marketing, etc… Nobody on the team had experience with or affinity for doing that.

To help us showcase the platform, Eliezer wrote the Bayes’ Rule Guide. We’ve went through several iterations of it over the course of a few months, tweaking features and improving retention metrics. The somewhat dense set of pages helped us test a few features easier. Lots of people read the guide and loved it, but it wasn’t obvious if Arbital format helped vs. Eliezer’s writing was good. I think people also didn’t appreciate the magic that happened behind the scene. (How do you communicate to a reader that they could have had a much worse reading experience but didn’t?)

Nate Soares helped us by writing the Logarithm Guide. We thought if we could produce good sequences like that frequently and post them online, we might slowly get traction. Unfortunately, it’s really time consuming to produce content that good, and there are just simply not that many people who can write content of that quality (and have the time to do it for free).

Here is what the front page looked like around that time. At the height of it, we had about a dozen regular users who would come and write a few pages every week. They enjoyed the small community we had and frequently hung out in our Slack channel. They wanted to write math explanations for themselves and their friends. I don’t remember how many readers we had, but it was around 50-200 /​ day, most of them redirected from Eliezer’s old guide to Bayes’ Theorem.

In August we raised a $300k pre-seed round. We had about 9 investors. Most of them invested because of Eliezer, but a few knew me personally as well.

Also around that time, it became clear to us that things just weren’t going well. The primary issue was that we completely relied on Eliezer to provide guidance to which features to implement and how to implement them. Frequently when we tried to do things our way, we were overruled. (Never without a decent reason, but I think in many of those cases either side had merit. Going with Eliezer’s point of view meant we frequently ended up blocked on him, because we couldn’t predict the next steps.) Also, since Eliezer was the only person seriously using the product, there wasn’t enough rapid feedback for many of the features. And since we wasn’t in the office with us every day, we were often blocked.

So, we decided to take the matter into our own hands. The three of us would decide what to do, and we would occasionally talk to Eliezer to get his input on specific things.

Reflection: Working with Eliezer was interesting to say the least. He certainly had a great overall vision for the product; one that I’m still astonished by to this day. He often had good insight into specific features and how to implement them. But sometimes he would get way too bogged down by certain details and spend longer on a feature than I thought was necessary. (In most of those cases he need things to work a certain way to solve a particular problem he had, but it was wasting our time because we were building something ultra specific to his use case.) This was especially painful for features that nobody, including Eliezer, would end up using.

Reflection: Eliezer also had a tendency sometimes to overcomplicate things and designs systems that I could barely wrap my head around. (I often joked that we would end up building a website that only one person in the world could use.) But then again, there were also many moments where a complicated, messy feature would suddenly click into place, and then it seemed obvious and simple.

Reflection: I’m tempted to draw a lesson like: never ever build a product you don’t understand yourself. But if I did that, I’d certainly miss a huge learning opportunity of working with Eliezer and leveling up my product and UX skills. So, instead, I think the lesson is: if you’re running the project, never ever do anything that doesn’t make sense to you. As soon as you start delegating /​ accepting things that don’t make sense, you muddy the water for yourself. Now the strategy has opaque components that you don’t understand, can’t explain, and sometimes actively disagree with. There is just no way you can move at the necessary startup speed like that, and you’re also not learning from your mistakes.

Reflection: This is especially true with respect to the overall strategy. Yes, maybe some paths are objectively better or easier. But if it’s not one that makes sense to you, if it’s not one you can execute, then you should take another path.

Reflection: Looking at arbital.com today, I’m actually still very much impressed with it. It’s a good piece of software, and if I wanted to write explanations, I think I’d be hard pressed to find a better website. Ironically, the large part of what makes it really good are all the features that it has.

Chapter 3: Pivot to discussion

It was clear that we couldn’t scale a community around math. So we decided to pivot. It wasn’t a clean and easy pivot; if I remember correctly it took us about a month of struggling and deliberating to decide that our current approach wasn’t working and then settle on a new one.

We decided to skip the Explanations part and go straight for the Discussion. We started build a new design around claims. A claim is a page with a proposition that users can vote on by assigning a probability estimate or by marking the degree of (dis)agreement. The idea was that people would blog on Arbital, and create pages for claims they discussed. People could vote on claims, and thus everyone could see where people mostly agreed and mostly disagreed. Claims could also be reused in other blog posts by the same author or other people.

We kept most of the original architecture, but remade the homepage. We also shifted the focus to the regional rationalist community. We did multiple user interviews. We did UI mockups. We talked to some rationalist bloggers and got some mild support.

One of my favorite artifacts to come out from that time period is this SlateStarCodex predictions page.

Reflection: At the time, I think the pivot decision was correct. And if we continued going with it, it’s possible Arbital would have become LW 2.0, though that wasn’t exactly our intention at the time.

Reflection: One thing we messed up during this time was diluting leadership. Since Eliezer was no longer in charge of product, the responsibility fell on all of us. This resulted in many many discussion about what to build, how to build it, down the minute details. Our pace really slowed down and it took us a while to patch it up.

Chapter 4: End of Arbital 1.0

In the beginning of 2017 I experienced my first burnout. There was simply no way I could work, so I apologized to the team and spent a month playing video games, which I desperately needed. This gave me the time, space, and distance to think about the project. When I came back, I sat down with the team and we had an extensive discussion about the direction of the company.

Eric and Steph wanted to stay the course. I no longer believed that was going to work, and I wanted to try a different approach. My biggest realization during my break was that people (and in this case specifically: most rationalists) were not actually interested in putting any serious effort in improving the state of online debate. While almost everyone wanted better online discussions, just like with math explanations, almost nobody was willing to put in any kind of work.

Furthermore, when we talked to a few rationalists directly, I just didn’t get the feeling of genuine helpfulness or enthusiasm. This was upsetting, because there aren’t that many big projects that the community does. So when I was doing Arbital, I guess I expected that more people would be on board, that more people would put in a bit of an extra effort to help us. But at best people put in minimal work (to satisfy us or themselves, I’m not sure). However, there was a limit to how upset I could be, because I very clearly recognized the same trait in me. So, while it’s still a sad state of affairs, I’d be a hypocrite for being upset with any particular person.

Reflection: I think the rationality community can produce great insights, but mostly due to individual effort. There are great posts, but they rarely lead to prolonged conversations. And you very rarely see debates summarized for public consumption. (No wonder, it takes a lot of time and hard work!) There are a few counterexamples, but I think they prove the point by how much they stand out. (Best recent example I can think of is Jessica Taylor mediating a discussion between Eliezer and Paul Christiano and then writing it up.) (And, of course, not only do those things need to be written, but they also have to be read! And who has time to read…)

Reflection: I’m pleasantly surprised by the currently active LW 2.0. I think this is some evidence against my claim, but overall, I still think that when it comes to building out more detailed models with explicit claims, especially when it involves working with other people, most people are not willing to put in the extra work. (Especially if their name isn’t attached to it.)

It was clear to me how to address this issue. People are willing to do what they are already doing. In particular: blogging. It didn’t seem that hard to take the software we had and really optimize it to be a better blogging platform, at least for some audience (like math bloggers). And it seemed obvious to me that we would at least get some users that way. The key difference from our path at the time was that instead of solving the Discussion problem and trying to get people to do new things, we’d simply focus on building a better tool for a thing people already do. Then once we had people on our platform, we could help improve the ongoing discussions.

Reflection: This was me finally channeling the “greedy” user acquisition strategy.

At some point during the debate we considered trying both projects in parallel, but at the end, Eric and Steph decided to leave. I’d take Arbital in the new direction by myself. (Huge thank you to Anna Salmon for helping to mediate that discussion. I’d say it went pretty well, all things considered.)

Chapter 5: Arbital 2.0

I spent the rest of 2017 working on Arbital 2.0. At first it was going very well. The vision felt very clear to me, I had my mind totally wrapped around it, and all parts of the strategy made sense. But for some reason, around summer of 2017 it became really hard to work. I spent a lot of the time trying to code, but being unable to. Even though intellectually I believed in the idea very much, my spirit was burned out /​ my System 1 just didn’t believe in the project. After struggling with it on and off for the remaining half of the year, I finally had to admit to myself that it just didn’t have enough momentum to succeed.

(The rest of this chapter is a Reflection.)

My best guess is that I was burnt out again. Even though I didn’t feel as bad as I did in January, the feeling of being unable to even touch the laptop was very similar.

For those curious and for those looking for a startup idea, I’m going to describe my plan for Arbital 2.0. In short, it’s Tumblr for mathematicians. You could use it as a blog, but it’s really a social network. What makes it radically different is the ability for one person to create and own multiple topic-centered channels. (One big issue I see with FB is that it doesn’t scale well with the number of friends. With most friends I only want to talk about certain specific topics. But FB is broadcast-to-all by design.) On Arbital 2.0, I would be able to post about improv, Rick and Morty, AI, scuba-diving, and all my other interests to different channels. People could subscribe to the channels they were interested in. So if you never wanted to listen to politics, you wouldn’t follow people on their political channel. (Or hide all posts with #politics.) Each channel could be grown into a community, where other people could submit their posts too.

I still think this approach is very likely to work:

• Write the software. (I have a 70% baked version.)

• Go to mathematicians and offer to host their blog on Arbital. (Why mathematicians? There is basically no good blogging software with great support for LaTeX. That feature plus a few others will convince many people to switch or at least to try Arbital.) When I cold emailed 100+ math bloggers, I got a good number of pretty enthusiastic responses. Path for 0 to 1000 users seems straightforward.

• Most math bloggers also blog about other things. This naturally will lead them to use the channels feature.

• Many people have topics that they can’t discuss on Facebook because they don’t want to spam their friends. (I’d make at least one Rick and Morty post a day if I could.) Arbital would be the perfect outlet for those. (I’d assign 20% probability that it’s possible to skip and succeed with this step directly without bothering with recruiting math bloggers first.)

• Then follow the original Arbital plan: Social Network → Explanations → Debate → Notifications → Rating → Karma.

• And, of course, put the entire thing on a blockchain to make crazy money in the meantime. ;)

It’s pretty clear to me that for Arbital to work at scale it has to be a social network. Part of why I don’t think most other paths will work is that social media ate all the free time. It’s not that people became lazy, it’s just when it’s a choice between spending another 15 mins on FB or spending that 15 mins creating and linking claims to your new blog post, most people will choose FB. (And while FB is the best example, the problem is more widespread, of course. Everything became more addictive.) This is why the new approach was to create a social network that’s better than FB and would allow you to manage your time and attention. And then from there we could actually put that saved time and attention to more useful things. (Although, I’m still sceptical that there are enough people who will have constructive debates to warrant all this effort.)

One reason I’m not pursuing this right now (aside from being burnt out with the whole enterprise) is that it no longer obviously helps with AI safety. If you recall, one of the assumptions was that if we did Arbital specifically for AI safety, the website wouldn’t get enough credibility. With some recent developments, I think that’s no longer the case. AI safety in general is more accepted, as well as MIRI’s research in particular. So, if I did any work in this space again, I’d, ironically enough, go back to the original vision of creating an explanation website for Eliezer and other people who wanted to write about AI safety. (But, actually, I think the bottleneck is good content and people actually reading it.)

Tidbits

What’s going to happen with Arbital?

I’m currently in the process of shutting down the company. All the software and IP is going to be turned over to MIRI. A few people expressed interest in having Arbital be open sourced, including one of our investors, so that’s likely to happen.

Arbital tech stack

Arbital 1.0: Golang on BE, Angular 1.0 on FE, MySQL DB, markdown editor.
Arbital 2.0: NodeJS on BE, React on FE, ArangoDB, SlateJS editor. (Much much better choices all around!)

How much do you think technical skill mattered on the margin?

A lot. This was a pretty complex project, so managing code complexity was important. We also needed to continuously optimize things to make sure everything loaded decently fast. We had to make sure there weren’t many bugs, because most things were user-facing. And being able to code decently fast helped a lot, since the amount of features we had to implement was fairly large.

Lawyers

I hated working with our lawyers. This was may be the most frustrating part of the entire project.

Lesson 1: work only with a team who is recommended to you personally by someone you trust and who has worked with them before.

Lesson 2: Ask how much something will take upfront. If the lawyer wants to spend more than an hour on anything, have them double check that you want it done. Have them send you a weekly report of time spent.

Lesson 3: Consider just not going with lawyers and using standard paperwork. Before Series A it just doesn’t matter and you can restructure things any way you want later.

Single founder

Being a single founder is not great, but there are actual reasons for it that in principle could be mitigated.

It’s unlikely you have all the skills. (Note that the situation is not that different if you have a co-founder, but they are very similar to you.) More important than the skills though, is your personality /​ inclination. Personally, I’d rather code than talk to users. So an ideal co-founder for me would be someone outgoing, who’d prefer talking to users than doing other things.

Not having someone to talk to day-to-day means you might end up with a tunnel vision /​ stuck /​ doing unimportant things /​ forgetting to take a step back /​ making basic mistakes. Having someone to talk to on frequent basis is important.

It feels bad when you don’t work /​ are stuck and the project doesn’t move forward. When you’re working with someone else, they usually make progress even when you don’t.

Responsibilities

It’s important that in each area there is a single person who is ultimately responsible for it. Product: one person. Business: one person. And, overall, for the company: one person. Assigning a responsibility to more than one person will significantly increase the communication overhead and slow things down.

Again, a heartfelt thank you to everyone who has participated in this adventure. It has been quite a journey and, at the end of the day, I wouldn’t take any of it back.

[edanm]

First of all, I want to join all the others in thanking you for the honesty and for the sharing.

I’m going to give a few of my views of this, as someone who has a fair amount of experience in “startup-land”. Some of this will be “criticism”, but please don’t take offense—it’s really hard to get these things right, and we all made and continue to make mistakes. And you seem to have gotten to some of these conclusions yourself—I’m writing this for the hypothetical other people who may want to start a startup, so making it general. Btw, really long comment, so sorry! :)

First of all, I’ll tell you what was by far the thing I miss most from your post—any talk about money. You’re building a for-profit company, and maybe I missed it, but I have no idea how you planned to make money off of this! I have no idea of the busines plan, at all. Even if making money isn’t really the goal here, unless you plan to live off donations, it should still be priority 1,2 and 3 for any company: You use money to solve most problems (pay to create content; pay to advertise; etc). You also use money as a good proxy for success. You also use money to fix problems like “burnout” by hiring people!

Secondly: I loved Inadequate Equilibria. But the whole “conversation about startups” was by far the weakest part of it, and the one part I think I actively disagreed with. (Also the part I know the most about: Gellmann amnesia anyone?) While I understand the concept of a grand vision, I think Eliezer and probably you are misunderstanding the idea of an MVP. Or at least, the way I think about it.

The idea is not, as Eliezer put it, to build a product that shows one specific workflow. For one thing, you don’t need to biuild a product. But more importantly, the emphasis is not on showing a complete workflow and seeing if people like it. The emphasis is on doing fast experiments. You need to figure out what assumptions you are making about what you’re trying to build, then test those assumptions. This is something you can often do with minimal work, by faking large parts of the product, for example.

One of the pushbacks to this view is that you might not know the assumptions, but that’s all the more reason to have fast experiments—you want to uncover which assumptions you’re making and don’t realize it, as soon as you can. If you think the only way to do this is to build a product over more than a year—you’re almost certainly wrong, except in very tech-heavy cases, which is not your situation.

In Arbital’s, some assumptions you had and could’ve tested:

1. You believed people will write content, for free. Easy to test—ask people to write it before the product.

2. You believed you had a superior flow for reading content. I’m still not sure I understand what that full flow was, but again, easy to test—ask Eliezer/​someone to write content, and make a static html site with all the “reading” functionalty, but none of the editing capabilties. You could reasonably make whatever workflow you imagine the reading experience to be with a week’s worth of hand-coding html/​css, or even a Wordpress site.

3. You believed you could get people interested in reading this material, and then doing… something? I’m not sure, since I didn’t understand the business plan. But let’s assume it’s “decide to read more material”. OK, easy to test—put one guide up, and ask people to sign up to a newsletter. Or donate money. Or something.

(I want to emphasize that, although I think I’m right, just the fact that Arbital failed doesn’t prove it. Building startups is hard and usually fails.)

Another issue that stems from lack of a business model—what kind of company were you trying to build? It seems pretty obvious, at least in retrospect, that this kind of company is a bad fit for a VC-funded startup. You are not trying to build somethign with minimal chance of success, but with the ability to become a billion-dollar company. I mean, I don’t think you were aiming for a billion dollar company.

But in that case, you should’ve never expected to raise money (and probably shouldn’t have raised money). And you should’ve made sure this is something that could be profitable relatively quickly, to continue supporting the development.

Lastly, I really got the sense from your post that you are all with a very engineering mindset, and very enamored by the beauty of a complex system, and by wanting to build something. Hell, you worked on this for a few years, wrote an entire post-mortem about it, and still I and others in this thread don’t even understand what you’re building! This is not a good sign—systems usually aren’t this complex, certainly not ones that are made to be used by actual people other than Eliezer :)

One more thing about community projects—we’re a community with a lot of developers. We see development projects everywhere. But the real strength of the community is not necessarily in that—if the bottom line thing we want, as a community, is more things like Eliezer-style explanation of hard concepts, that’s hard enough—we should make the journey to creating that content incredibly simple, and while that is arguably what Arbital was, I’d say that “let’s spend a few years to develop a new software platform” is a huge burden. Much better to use pre-existing stuff, IMO. Let’s make a rationality-Stack Exchange. Or a rational wiki (or not :) ). (Not to crticize too hard because I’m not that much in the community and don’t know the details, but I kind of wonder the same thing about LessWrong V2.0 - do we really need to rebuild forum software from scratch just for us? Is that really where we should be spending our community’s talent and efforts?).

To conclude: building starhtups is hard. Building consumer startups is much harder. Building consumer startups that are marketplaces is really really freakin hard. You tried and failed, which is a shame, but you seemed to have learned a lot from this, both about startups and other things (based on Inadequate Equilibria, I think Eliezer hasn’t learned the lessons I would’ve learned). So kudos for trying, kudos for putting yourself out there with this post, and in general, good job!

[ChristianKl]

From what Eliezer wrote at the time he did think that it will create huge economic value and make massive profits. If a substantial number of people learn skills by reading on Arbital that would have allowed to make money by running ads and also in other ways.

[alexei]

Thanks for the comment. Yeah, we definitely planned to make a lot of money. But I think the steps from what we were building to where we would be making money were too indirect /​ too far.

[TAG]

still I and others in this thread don’t even understand what you’re building!

On a related note. a 55 page spec should not leave people basically in the dark.

[Qiaochu_Yuan]

First, I want to echo that I’m extremely grateful for this writeup, and also for your hard work on a plausibly important project.

I was one of the people approached in 2016 to write math content. I said I’d think about it but never ended up writing any (aside from, IIRC, a small handful of minor edits to existing pages), and I don’t remember if I gave a detailed explanation of why I didn’t feel excited about writing content on Arbital, so for what it’s worth, here are some extremely belated thoughts about that. I want to contrast Arbital with Math.StackExchange in particular, where writing content is if anything too easy and addictive for me.

• First and maybe most importantly, answering questions on math.SE involves a fast and satisfying social exchange. A person asks a question, I answer it, and then I get various social rewards, namely upvotes or comments, which are often of the form “thanks for this clear explanation!” or similar. It’s easy to get a sense that I’m helping people, and it’s nice that I get clear social credit for providing that help. The fact that I’m answering a question also means I don’t have to pick a topic to write about (this is part of what’s preventing me from writing top-level LW 2.0 posts), and I can also tailor my explanation to what the questioner seems most confused about. I got the impression (I don’t remember how accurate this is) that writing an Arbital explanation would be too similar to writing a Wikipedia article, which I’ve never been excited about: I don’t get social credit for helping, I don’t know who is being helped, I have to pick the topic, and I don’t know who to tailor my explanation to.

• Looking back, I was unsatisfied with the whole concept of collaboratively writing a long modular sequence of explanations. There were roughly two ways this could go and I disliked both of them for different reasons.

  • Way #1 was that I’d mostly write a few pieces of such a sequence; I disliked this because 1) I didn’t want the comprehensibility of my explanations to depend on the comprehensibility of other explanations I hadn’t vetted, and 2) I didn’t want to have to fit into a particular narrative or frame from other explanations if I thought I had a better one.

  • Way #2 was that I’d mostly write such a sequence myself; I disliked this because 1) it takes cognitive effort to hold the first N pieces of a long explanation in working memory when modeling a reader reading the (N+1)st explanation and I wasn’t willing to do this casually, and 2) I didn’t like the idea of writing something this long for an abstract audience as opposed to a particular person or people because I didn’t feel like I had enough to go on as far as modeling where the audience is likely to be confused, etc. Having to model a variety of possible readers was also cognitively effortful and I wasn’t willing to do that casually either. The experience would have felt noticeably different for me if I was asked to model a specific set of readers, e.g. “please write an explanation of logarithms for Alice, then for Bob, then for Charlie”; then it would have felt more like answering a sequence of related math.SE questions.

To the extent that I didn’t explain this to Eric when he asked me in 2016 I can only plead that in 2016 I was less good than I am now at noticing and articulating ways in which I’m unsatisfied or annoyed by something; also I was to some extent responding to mild perceived social pressure to be enthusiastic about the project.

[alexei]

Thanks for the response. After reading it, it’s now even more clear to what extent collaborative explanations is just not a thing that can easily work.

[Eugen]

The obvious question is how is it even possible that Wikipedia works at all? If Wikipedia didn’t exist in our universe, we would now be tempted walk away from this with a high probability estimate that this concept is simply impossible to pull off due to the various reasons mentioned, yet here we live in a world where Wikipedia is clear evidence to the contrary, and to my knowledge it suffers from many problems you and Qiaochu_Yuan mentioned above. Are we to conclude then, that the sequential nature of the arbital content is the crux here?

As we all know, you can almost always dig up something on what could be considered the most obscure niche topic. So what is the core appeal for the vast number of content creators? Is it simply that Wikipedia is recognized as the internets “centralized encyclopedia” and contributing to it feels so high status that ones total anonymity is not perceived as a huge issue? That would not explain how it got to where it is today, how did Wikipedia bootstrap itself to where it is now?

[abramdemski]

Post-hoc analysis—Wikipedia relies on a relatively small number of people who have the unusual motives/​dispositions necessary to write for it, so it doesn’t have to cater to the masses or even the typical elite. However, that doesn’t explain how enough people got interested in the first place. Maybe it had to do with there being fewer alternative venues back then. Also, I’m not so familiar with the history, but I am guessing it grew out of a fairly vibrant community of wiki users. Arbital, on the other hand, tried to bootstrap in relative isolation. (Sadly the vibrant community of LWers didn’t serve the same function, likely because the use-case for Arbital 1.0 was too different from the use-case for LW.)

[Kenny]

I’m sad I didn’t think of it before, but hosting LW content on Arbital might just be amazing. In particular, I’d like some kind of way of keeping track of networks of related claims with community ratings and counter-claims clearly visible.

[alexei]

Sound correct to me. Also, I think it’s much easier to start /​ contribute to a page on Wikipedia. Arbital’s pages were trying to be educational and readable, which, I think is a higher bar.

[ChristianKl]

Wikipedia can often trigger a “this is wrong on the internet” reflex that gets people involved.

[Qiaochu_Yuan]

Yeah, I don’t really understand how Wikipedia got to where it is today. I think it is mostly edited by people very different from me; if I had to take a wild stab, much more autistic (and bless them for that, to be clear).

[ShardPhoenix]

When I contributed a little in the early days I think it was driven by desire to promote the things I was interested in (mostly video games this case). I’d guess the same goes for historians etc writing the more serious articles. It helped that standards for things like NPOV and citations were a lot lower back then—I don’t think the project would have gotten far if everything had to be cited up the wazoo like it is now. (Now that Wikipedia is established as being important people are willing to put a bit more work in).

Although I’m not autistic as such I do suspect there’s a connection between the above and the stereotypical autistic desire to excessively talk about one’s topic of obsessive interest.

[cousin_it]

Yeah. I think it’s possible to create an addictive website that spits out good explanations, but it’s mostly an incentive problem and it seems like Arbital didn’t have a good story for incentives.

[Eliezer Yudkowsky]

I designed a solution from the start, I’m not stupid. It didn’t get implemented in time.

[Markk116]

I’m very curious how you solved the insentives problem, would you mind detailing it? Alexei mentioned that you already did the write-up, so even a link to your rough-draft would satisfy me.

[ChristianKl]

The fact that I’m answering a question also means I don’t have to pick a topic to write about (this is part of what’s preventing me from writing top-level LW 2.0 posts)

Is that a true objection? If so, maybe you can find somebody who tells you what to write?

[Jan_Kulveit]

I believe I can add something useful to this conversation.

Background: I noticed Wikipedia when it was a relatively new project. I was one of the first registered users of Czech Wikipedia (# 7), one fo the first builders of the Czech community, registered the Czech domain, and gradually worked in different functions suchas an administrator, member of the arbitration committee, checkuser, etc … I was also a founding member of the Czech Wikimedia Chapter.

Since then, I have been following various attempts to compete with Wikipedia (does anyone remember e.g. Google Knol?) And so far in all cases I expected the projects will fail.

Why? Because IMO essential factor behind the success of Wikipedia is the “community design” that is in the background, and which provides incentives to content creators. Common lore it something likeeveryone can edit Wikipedia, and the encyclopedia is written by mostly random crowd of site visitors who click the edit button. That’s complete nonsense. Wikipedia was written mostly by a dedicated community of volunteers with complex internal organization, hierarchies, social rewards, and the possibility of gaining power.

Much of what I did at the begining cz.wiki was translations of various documents about community norms, implementation of different processes, feedbacks, dispute resolution, etc. The community design goals are various, e.g. you want positive feedback loops, you dont want editors to bite each other, you want the community to be meritocratic, and so on. It’s far from obvious how to design such system.

Of course, with most wikipedists, somewhere in the background is an altruistic motivation to help with the aggregation of human knowledge and create something like Encyclopedia Galactica. But on a day-to-day basis, what helps keepeing peeple motivated is working with other dedicated people, receiving feedback, beeing able to see others interacting with your edits and improving further, and even some forms of conflicts . Also valuable editing leads to increasing your weight in the community, you can gain various social goods, responsibility, various functions, and of course power.

Btw the power of senior Wikipedia users is not negligible at all. If we simplify it to counting money, you can imagine what is the value at stake depending on the content of the first page of Google results on many topics, and how much power Google has controlling it. Wikipedia has less influence, but still a lot of influence.

So to summarize, what I see in this portmortem (thanks for wrting it!) from my perspective is almost complete lack of “community design” and thinking about editor motivations. (Community design is different from user aquisiton.)

[alexei]

We did some work on the community design. That’s what Eric Bruylant did part time (slack channel, writing guidelines, etc..). He worked with the Wikipedia community (and other forums) in the past, so he certainly had the right experience. But overall I agree with your sentiment.

[Qiaochu_Yuan]

Thanks Jan, this was useful. I had a vague sense that Wikipedia had a bunch of community stuff in the background but it’s cool to see it explicitly given as an important factor behind how it works; that gels much better with my StackExchange experience.

[Adam Zerner]

It seems that Eliezer had plans to incentivize content production, but that these features never got built. However, I do find it curious that those features were deprioritized enough to be pushed back years into the journey.

[Raemon]

Joining the chorus of apprecation—it sounds like writing this would have been both intellectually and emotionally exhausting (or, at least, it would have been if I had written it)

My question for you is, of course, which features from Arbital seem relatively easy and worthwhile to port over to LW2.0? I’m still not sure I grok the entirety of the vision and how everything fit together, but it seemed like at least some of the features would be fairly easy to implement without relying on the entire vision to be useful.

I guess also curious, for curiosity’s sake, in “what features wouldn’t work, because they required too much on the tangled web of interconnected pieces?”

[alexei]

Two features I miss the most are greenlinks (hover over a link to see summary) and claims (vote with probability /​ agreement).

But I think this question should be answered by LW community needs.

Well, trying to build a system that will dynamically link pages together to form a sequence based on requisites would be hard. But I think basically all other features are very modular.

[habryka]

I also really want both of these, and have needed to restrain myself from just building them, since I do think for now we should focus on the core experience of the website (and you know, finally actually move to LessWrong.com and properly import all the old content and stuff). But I definitely want to have both of them in the long run.

[Adam Zerner]

If you want to build them so badly that you feel like you need to restrain yourself from doing so, I’m sure you’d receive a nice morale boost from just building them now. And morale matters. I’m not sure if the benefit of morale outweighs the benefits of priority, just offering it as something to consider if you haven’t already.

[John_Maxwell]

+1, I’m increasingly convinced that the best strategy for productivity is to juggle 3-5 projects and just work on whichever seems most appealing at any given time. See structured procrastination. Pro tip: If you start having ideas related to an important topic like AI safety, that’s a blank check to procrastinate on anything else.

[Adam Zerner]

Huge “yesssss” to greenlinks.

Claims I’m not so sure about. People do, and want to, upvote for different reasons. Stating agreement is surely one of them, and claims sounds like a good way to address that need. But what about other needs, like stating appreciation or inspiration? Do we have different types of voting for the different use cases? Does that become feature overkill? I don’t mean to imply any answers to these questions, just that I personally don’t feel a sense of excitement and confidence about that feature.

[Qiaochu_Yuan]

My gut intuitions say that this is overkill, but on the other hand I use Facebook reactions all the time and get useful information out of them.

[Vanessa Kosoy]

Thank you for writing this post-mortem! I think that if we want to start “winning” as a community is it very important to analyze the times we don’t win.

I got the impression that Yudkowsky had a fascinating grand vision but lacked experience /​ domain knowledge in building software products, whereas the rest of the team might or might not had sufficient experience /​ domain knowledge, but if they did, they were not assertive enough in applying it. More generally, I suspect that experience /​ domain knowledge are somewhat underrated in the community compared to generic rationality skills (and I say this as someone who believes generic rationality skills are very important).

Aside of that, I really like the idea of a social network with different channels. Relatedly, I thought (and still think) that it might have been better to make LW 2.0 more like a social network.

In any case, I feel that, although the universe doesn’t reward effort in itself, the community should, and I personally believe the Arbital team made a laudable effort to improve the world, even if it wasn’t as succesful as we might have hoped.

[alexei]

… but if they did, they were not assertive enough in applying it.

… experience /​ domain knowledge are somewhat underrated in the community compared to generic rationality skills

Yes to both.

And yes, I’d love to see LW 2.0 execute my plan and become a social network. (They already did the first few steps; just instead of math, they did rationality.)

[Chris_Leong]

“Aside of that, I really like the idea of a social network with different channels.”—I’ve had this idea for years, but sadly never pursued it. Anyway, such a network already exists. It’s called Google+. PInterest works like this as well.

[TAG]

More generally, I suspect that experience /​ domain knowledge are somewhat underrated in the community compared to generic rationality skills

Much more than somewhat!

[cousin_it]

Thanks for writing this!

The most striking part is that there were cheap ways to test demand for your product without building it, and you even gathered some data which suggested low demand—but you didn’t trust the data or look for more data. I guess founders are just biased toward building things, otherwise they wouldn’t be founders!

[zulupineapple]

The most striking part is that Eliezer then went on to criticize this perfectly good idea in his Inadequate Equilibria. At the time I read that, I had no idea, but it now seems that modesty was the right thing to do.

[alexei]

Which version of product are you talking about specifically?

Also, part of the reasoning was that if we had a functioning product, we could try many things with it. (In practice, we only got to try a few.)

[cousin_it]

It seems to me that all versions of Arbital relied on the hypothesis that many people would create content, but every time you looked at user behavior, you got evidence against that hypothesis. And you paid a lot for that evidence. It would’ve been much cheaper to ask people if they would create content.

[Eliezer Yudkowsky]

The vision for Arbital would have provided incentives to write content, but those features were not implemented before the project ran out of time. I did not feel that at any point the versions of Arbital that were in fact implemented were at a state where I predicted they’d attract lots of users, and said so.

[cousin_it]

Interesting, any chance you could describe it?

[ChristianKl]

Given that the project did have time to pivot and try something different, it seems to me as if time was there.

It sounds to me like the main problem was communication and agreeing on a common vision?

[Taymon Beal]

Thanks for the informative writeup.

I already said all of this on Facebook, but just to reiterate:

• I believed from the first announcement, and continue to believe, that much of the value of Arbital as it exists is in the software itself. (By comparison, if Wikipedia stopped existing, MediaWiki would still be important and valuable.)

• I, personally, want my own Arbital instance that I can use to write about EA donation opportunities. (I think Malcolm Ocean has said he wants one too.)

• If and when it gets open sourced under any of the usual open source licenses, I will contribute documentation, automation scripts, and/​or settings cleanup as needed to make it self-hostable.

[Adam Zerner]

Reflection: I think the correct thing to do would have been to create detailed UI screens. Then print them and show them to people (and Eliezer). This probably would have taken a month or two, but it would have been worthwhile. The reason I never got around to it, aside from the ugh-field around doing UI mockups, was because it always felt like in a month or two we would be done with the MVP.

This stands out as my favorite paragraph of the post.

I understand where Eliezer is coming from about building a product that is the best in the world for that particular workflow. An MVP is a test. The point is for you to be able to say to yourself, “Uh oh. People don’t like the MVP. That (probably) means that they won’t like subsequent versions either, and so I don’t need to waste any more time working on this.” If the version you release doesn’t allow you to say that to yourself in the case of a failure, well, I don’t see that it’s a useful test. And if it’s not a useful test, then what’s the goal of the release? Sometimes the goal is just to get feedback on usability and features/​product stuff, and I think that that is often a valid goal.

So, I’m firmly on the side of “make sure your MVP is actually viable, and thus a useful test” rather than the side of “release early, release early, release early!”. But at the same time, I think it’s important to really look hard for ways to release your MVP earlier (while still having it be viable) as opposed to just conceding, “Oh well, it’s just going to take months of development before I can test my hypothesis”. I love the idea of making UI mockups, and demoing them to potential users. If people really, really want this, then they’d get excited at the demo and say, “Yes, yes, yes! I want this! Build it for me!” Right? Perhaps not. Perhaps my model of users is flawed.

I’d also like to highlight the comment you made about always feeling like you’ll be done with the MVP in a month or two. I’m working on a software startup too right now, and I make the same mistake. It started out as a project to help me develop my coding skills so that I can level up and start freelancing. It started out as a Rails API with no front end. Then I figured, “what the hell, let’s build a little front end”. Then I started to think I was on to something that could make me money, and that it’s worth spending a month or two improving the front end (partly to improve my coding skills). Then I figured I should learn ReactJS (I had spent the past year self-studying CS, and before that, Angular was my thing), so I redid the front end in React. Then I decided that I don’t like React, and that I do like VueJS, and so I redid it in Vue. At that point I was really feeling like it could make me money, and that it’s worth another month or two to improve it further. Then I realized that hitting my server’s API takes too long (because of network latency), and so I rewrote all of my backend code on the front end. From there it was just a lot of me saying to myself, “it should only be another month or two”. At the end of the day, I’ve spent over a year on a project that I never really intended to spend more than 3-4 months on, and I still feel like I only have another month or two before I release and can test my hypothesis.

Anyway I really just want to say that this planning fallacy failure mode is real, and that it seems especially harmful for people working on startups, because agility is so important with startups. (Also, I initially felt too embarassed to share the extent of my planning fallacy failures, but your honesty and disclosure inspired me to do it.)

[Ben Pace]

Failed project reports are really awesome, I don’t get a lot of data about projects that get this far along and yet fail. Also the reflections were super helpful, in knowing what seems salient to you in retrospect. For these reasons, I’ve curated the post.

If at some point in the future you’re able to write in more detail on the hypotheses about UI design, intellectual progress and collaborative explanations that arbital was trying to test (and the failure of which is evidence against), that would be facinating to a further extent.

Also, thanks for the suggestion regarding giving math bloggers a nice place to blog that has LaTex, I’ll think if there’s something to try with LessWrong to that aim anytime soon.

[alexei]

If I heard correctly that AF forum is moving to LW 2.0, you’ll have to solve the math blogging problem. ;) And with the current features you’re already 50% there. (Assuming they are working well, which right now it doesn’t quite look like that.)

[JenniferRM]

Thank you for the writeup! I’ve long had a distant impression of Arbital as being some kind of “mindmapping prediction social thing” and now that I’ve heard the explanation of its iterating vision I think maybe my model of it might be “Alexei and Eliezer’s Memex or Xanadu”.

This updates me a bit in the direction that something like Arbital will exist in the future and be a big deal, and it will probably make more progress by exteme attention to (1) the microeconomics of users and their existing preferences and their desire to have property they seem to control and (2) compromising on the overall “economic architecture” of the system such that it does not actually bring about the full utopian societal transformation it initially promised.

[alexei]

Fun fact: originally Eliezer called the project Zanaduu (a play on Xanadu-doomed).

I’ll bet that parts of Arbital will show up across various products (and I’ve already seen some), but I would be very very surprised if we get something that has the entire package in the next 5 years.

[ChristianKl]

I sometimes felt to me like Artibal didn’t care about attracting users at all. While Eliezer wrote posts explaining various issues about AI alignment on Arbital nobody linked to the explanations on LessWrong.

If I google: site:lesswrong.com link:arbital.com -site:wiki.lesswrong.com I don’t see any links from LessWrong to Arbitral (there are some on the LessWrong Wiki but that doesn’t get much readership). The same goes for LW 2.0.

There was a single post asking: “Eliezer wrote those great explanation on Arbitral, why is nobody linking to them” on LessWrong.

[ESRogs]

While Eliezer wrote posts explaining various issues about AI alignment on Arbital nobody linked to the explanations on LessWrong.

In case anybody reading this is curious about those AI alignment posts:

https://​​arbital.com/​​explore/​​ai_alignment/​​

(note: loads slowly)

[Pongo]

Just to say, these are amazing. I would rate them above Superintelligence, or indeed almost any other resource, for increasing someone’s concrete understanding of AI safety

[cousin_it]

At least this post of mine had a link to Eliezer’s Arbital writeup on “rescuing the utility function”. I’d be sad to see it go, there’s some damn good writing in there.

You’re right that they didn’t do enough to attract users though. I only found it by accident after someone on IAFF mentioned that Eliezer is writing stuff on Arbital.

[Ben Pace]

You’re right that they didn’t do enough to attract users though.

I want to point out how trivial it would be for Eliezer to get way more users/​readers on Arbital than he has gotten, just by sharing the best essays on Arbital once a month on facebook. He would’ve gotten like one or two orders of magnitude more readers. I’m confused you think Eliezer was missing this by accident, rather than fully aware of what his options are and then deciding to hold off on bringing users to his as-yet-unfinished product.

[cousin_it]

At the time I thought it was an under-construction CFAI kind of thing and he was avoiding too much attention from randoms, but your explanation makes sense too.

[Kaj_Sotala]

Thank you for the post from me as well!

For those curious and for those looking for a startup idea, I’m going to describe my plan for Arbital 2.0. In short, it’s Tumblr for mathematicians. You could use it as a blog, but it’s really a social network. What makes it radically different is the ability for one person to create and own multiple topic-centered channels. (One big issue I see with FB is that it doesn’t scale well with the number of friends. With most friends I only want to talk about certain specific topics. But FB is broadcast-to-all by design.) On Arbital 2.0, I would be able to post about improv, Rick and Morty, AI, scuba-diving, and all my other interests to different channels. People could subscribe to the channels they were interested in. So if you never wanted to listen to politics, you wouldn’t follow people on their political channel.

For what it’s worth, Google Plus has basically this under the name “Collections”.

[Qiaochu_Yuan]

Maybe relevant: I periodically hear jokes about nobody using G+ but actually I know a few people who are still using it to this day and they are all mathematicians except for gwern.

[Matt Goldenberg]

And calling gwern a non-mathematician almost feels incorrect… even though it’s correct :).

[Chris_Leong]

Someone needs to clone Google Plus and give it a different name. Or Google could just give it a new skin, tweak a few features and relaunch under a different name...

[ChristianKl]

All those features of filtering are only helpful when there’s enough posts of people one follows in the first place. That doesn’t happen by starting a new Google Plus clone or a new skin.

[alexei]

Huh!! 2015, no less. I’ll check them out.

[Said Achmiz]

Having read the post, Alexei’s comments, and now a couple of Eliezer’s comments, it seems to me that the takeaway is this:

Arbital has not been tried and found wanting; it has been found difficult [to make] and not tried.

Well… does anyone else think that it might be valuable to give it another go? It sounds like some critical parts of Eliezer’s design never actually made it into any product that was ever made, much less user tested and much much less released to any significant population of users, so really, we still don’t know much, if anything, about whether the idea would succeed.

Is the thing inherently difficult to make? Some of what Alexei said suggests this, and no doubt there’s some amount of inherent difficulty. On the other hand, many of the difficulties Alexei describes, that plagued the development process, seem eminently avoidable.

Eliezer, do you still think that Arbital (i.e., your idea for it) would be a good thing to make?

[alexei]

Yes, there is a good case to be made that Eliezer’s vision hasn’t been fully tried. But I also think it’s impossible to try it because the exact design is locked inside Eliezer’s head, so he would be the bottleneck. (That is, unless you found someone who thought like him and could be on the team full time. We have tried to find a person like that, but couldn’t.)

I maintain that someone doing their own project in this area would be a better bet. And they can take features /​ inspiration /​ overall direction from Arbital.

[Said Achmiz]

But I also think it’s impossible to try it because the exact design is locked inside Eliezer’s head, so he would be the bottleneck.

Well, Eliezer has access to a keyboard, and a computer, and the Internet, doesn’t he? The design need not remain locked in his head!

(“But Eliezer is too busy to type up all of his thoughts!” Well, you (Alexei & the Arbital gang) kept design notes, right? Why not publish them? Or are you keeping them to yourself because you still have plans to develop a project on their basis in the future? In any case, here’s another possibility: have Eliezer explain the thing to someone (perhaps verbally), record it, then publish that!)

In short, this seems like an eminently solvable problem; I find ‘impossible’ to be a strange thing to say of it!

[John_Maxwell]

I’m confused why there is such a focus on understanding Eliezer’s ideas, given that everyone seems to agree the first online community Eliezer had custom-built for him (LW 1.0) was a failure. Having someone custom create an online community for you is an incredible luxury. I suspect there are a lot of other people who have detailed models of how online communities work and/​or large internet followings that could be induced to migrate to a new community.

I do think it’d be valuable if the Arbital team released their design documents so others could draw inspiration—I just don’t see any compelling reason to treat Eliezer as an authority for the purpose of making decisions.

[habryka]

It seems very weird to me to call LW 1.0 a failure. Sure, nobody maintained it and so it slowly declined, but it still gave rise to a pretty massive and active community and was the hub of a lot of excellent writing for quite a few years (e.g. Scott Alexander, lukeprog, etc.).

[John_Maxwell]

Yes, I agree all that was valuable and important.

The way I think about online communities, there are two important inputs: There’s your initial endowment of users & attention, and there’s the skill with which you design the community culture, rules, software, etc. These interact in the form of an exponential function.

If you do a great job of designing community software, your community can grow exponentially in to something massive/​beautiful like Facebook, Wikipedia, etc. Online community success is power law distributed.

If you don’t do a great job, you get exponential decay instead, and the community gradually fades away. Eliezer put a lot of effort in to the exponent for Less Wrong 1.0: A custom platform was built to Eliezer’s specification, and he wrote an entire sequence about the sort of culture he wanted. But none of that prevented exponential decay—even while the rationalist community itself expanded! Instead, LW 1.0 turned in to a site Eliezer himself didn’t want to use.

It’s possible that LW 1.0 would have thrived with more maintenance. But I think it still counts as evidence against the idea that Eliezer has unusual ability as an online community designer. First, if people get enthusiastic enough about an online community, some of them will step up to maintain it. So “lack of maintenance” is not cleanly separable from other trends toward decline. Second, by saying that LW 1.0 failed due to lack of maintenance, you’re effectively saying that the Eliezer strategy of carefully figuring out how the community should work from first principles, without any empirical feedback, is inferior to a more hands-on strategy of keeping your hands on the wheel and seeing where things lead. This also appears to be one of Alexei’s takeaways.

Again, I’m not saying we shouldn’t listen to Eliezer’s ideas. If I was creating an online community, I’d love to take a look at the Arbital design document. But I would not follow Eliezer’s advice if it didn’t make sense to me, and it seems Alexei also reached this conclusion. Indeed, I see this as a big takeaway of Inadequate Equilibria: if the advice of high status people doesn’t make sense to you, consider not following it.

[habryka]

As someone who has worked at Arbital (though I joined after Alexei went on break, so he might think differently, and I never talked to Eliezer directly during my time there), my read is that the things that Eliezer did write down were not comprehensive enough to build a coherent model of his internal model. Multiple long conversations with Eliezer did also not seem good enough to get someone to build a strong enough model so that they wouldn’t very quickly propose things that Eliezer thought were very bad ideas. Having the notes and conversations publicly, allowing more people to study them, might help here – but I am currently skeptical that that would be enough for anyone to successfuly extract Eliezer’s models.

[Said Achmiz]

I see, thanks. Does anyone other than Eliezer have a good idea of what Eliezer wanted (wants?) to build?

[habryka]

Nate Soares had some strong opinions, and I generally expect him to have good models of Eliezer, but he is not any less busy than Eliezer is.

[alexei]

What habryka said. Basically you’re totally underestimating the complexity of the project and how granular and specific things get if you’re to build them in a way Eliezer would approve.

[Adam Zerner]

Later that year Eliezer wrote a 55 page document describing Arbital and how and why it was different and necessary.

Given that Eliezer, you and habryka are saying what you’re saying, out of respect for all of you, I truly do believe that you are right. But it is hard not to be curious when a 55 page document was released and about three years worth of work had been done (meaning that there’s notes, designs, code, and mental models that you and the other people who worked on the project have).

I can sorta grok the idea that it wouldn’t be possible to get every small detail exactly the way Eliezer envisions. But what about using the big document plus everything else for the broad (and medium) strokes, and filling in the details by using ones judgement? The details won’t get filled exactly the way Eliezer envisioned, but I’d think that it’d be a decent approximation. And even if it isn’t such a great approximation, the broad strokes would still be there, right? To wrap up this thought: I’m having trouble grokking why it would be essential to get all of the small details exactly the way Eliezer envisioned.

[alexei]

I sympathize. It’s a giant and weird project the likes of which the world has not seen in a while. If I wrote down how to implement just what we built so far so that someone could read it an unambiguously translate it into the current product, I think the document would be around 200 pages. And what we implemented was may be ~15% of Eliezer’s full vision that he was describing in his document.

By the way, we followed Eliezer’s direct vision for only 1.5 years. Then we took matters into our own hands and the design went elsewhere.

Turns out it’s hard to get the broad details right too. It’s basicly hard on every level.

If it’s not according to Eliezer’s specification, then it doesn’t have Eliezer’s “magic touch”. I think if you’d ask Eliezer, he would tell you that the feature you built (or the whole product) won’t work as well or at all.

[TAG]

if you can’t unlock ideas by writing a clear spec, you shouldn’t even be trying to run a software project. That woud be like hiring someone who can’t code to develop.

[Liron]

Thanks for the candid write up!

I’d make at least one Rick and Morty post a day if I could.

Given your thinking that Arbital prioritized engineering too much over acquiring users (aka validating demand), consider testing this claim of self-demand before investing effort toward it.

[romeostevensit]

People underestimate ontology fatigue/​ontology loading/​preexisting ontology complexity etc in UI/​UX design. More complicated people have more complex ontologies in their workflows, it’s not very surprising that a design hyperoptimized for one is not very appealing to others. Also, ‘flexible ontology’ services with complicated UIs for the user to roll their own are generally a shitshow. Better for the platform to be popular enough with its base implementation that users start writing custom scripts for enhanced functionality that you then add in/​rewrite. See popular online games and social networks. Chain stores exist partially because of the cognitive overhead of ontology loading.

[Said Achmiz]

Great writeup. I have been wondering for a while what was/​is going on with Arbital, so I really appreciate this post.

There is basically no good blogging software with great support for LaTeX.

FYI, the software I use for my blog (PmWiki with BlogIt) has excellent support for LaTeX (I also have embedded Sage [computer algebra] cells, GraphViz, and gplot). It’s basically everything a mathematician’s blog needs.

[alexei]

Yes, there are components one can put together to make it all work well. But there is nothing as simple and as good looking as Medium.

[Adam Zerner]

But there is nothing as simple and as good looking as Medium.

Agreed. Although I’d like to also give a shout out to https://​​trix-editor.org/​​.

[habryka]

Do you have experience with trix? I considered it for a while for the LW2 editor, but wasn’t sure whether it wasn’t modular enough to build custom UI stuff on that I would definitely want later, but I am also not super happy with draft.js, which is what we are using right now.

[Adam Zerner]

Actually, not too much. Maybe I shouldn’t have given it a shout out. I used it in one small side project, and it was easy enough to plug in, I think it’s really well designed and has great features, and in my mind, Basecamp has a really great reputation.

But I don’t know anything about building custom UI stuff on top of Trix. If it’s difficult to do, and important to have, I could totally see Trix not being the right choice for LW 2.0. However, if the custom UI stuff isn’t going to come until much later on, perhaps it would make sense to use Trix in the mean time and then transfer over to something else when you’re ready for custom UI stuff. Although that depends on how difficult the transferring over would be. I dunno.

[Said Achmiz]

“Good looking”, of course not, because Medium can afford, and has, a team of designers working on their site’s appearance. With an easy-to-mod platform plus at least one good designer, you can get pretty close to Medium levels of good looks (or even better, as Medium is constrained by its nature to make certain questionable or outright poor design choices).

On “simple” I have to disagree. There’s nothing complex about the setup I describe (from the writer’s perspective). (Of course it needs a good admin supporting it—as does any platform with non-trivial capabilities.)

[alexei]

Of course it needs a good admin supporting it

Yup, that’s a nonstarter for most casual bloggers.

[Said Achmiz]

I meant the platform needs a good admin, not each individual blog. Does Medium not need admins? Does Wordpress (wordpress.com, I mean, not self-hosted Wordpress installs) not need admins? Does Arbital not need admins?

What I’m saying is that if, hypothetically, I wanted to create a platform for other people to have blogs on (who needed LaTeX support and so on), I wouldn’t need to develop new software for it, because such a thing already exists (and is very capable).

[Jes]

Well, it’s not dead. It taught my daughter and I how to reason using Bayes’ Rule, something I had completely given up on understanding until I read the fantastic guide from Arbital.

Maybe it didn’t become a cool trend that people clamor over, but it probably helped other people with similar questions!

Here’s the link if anyone else wants to learn the same: https://​​arbital.com/​​p/​​bayes_rule/​​

[t3tsubo]

For those of us who are new to the community, can you have a writeup at the start of the explaining what Arbital is? I got to Chapter 2 in the article and I still had only a vague idea what it was (is?) and the website itself doesn’t even have an ‘about’ page or explanation other than ” Arbital is a hybrid blogging and wiki platform. ”

[TAG]

There was perhaps never a clear endpoint. And projects without clear endpoints rarely succeed.

[TAG]

7 yeas ago:-

I think jimrandomh is slightly too harsh about Flare, the idea of using a pattern-matching object database as the foundation of a language rather than a bolted-on addition is at least an interesting concept. However, it seems like Eliezer focused excessively on bizarre details like supporting HTML in code comments, and having some kind of reference counting garbage collection which would be unlike anything to come before (even though the way he described it sounded pretty much exactly like the kind of reference counting GC that had been in use for decades), and generally making grandiose, highly detailed plans that were mostly impractical and/​or far too ambitious for a small team to hope to implement in anything less than a few lifetimes. And then the whole thing was suddenly abandoned unfinished.

Plus ca change...

[ChristianKl]

Source

[Adam Zerner]

Reflection: Another struggle was: “we need users to make sure we are building things correctly” (Alexei) vs. “I can tell when we are building things correctly, I can get us users as soon as the product is ready” (Eliezer). Unfortunately, we never got the product “ready” enough to test Eliezer’s claim. I think it would have a taken a long while to get there. But, given how things ended up, it’s possible that would have been a better path.

This makes me think that you and Eliezer weren’t the right partners for each other. From his perspective, why go through with the project with a partner who hasn’t agreed to commit to building out the product enough to test his claim? From your perspective, why work with someone with a different vision regarding talking to users? I suppose the answer is that it is difficult to remember to ask all of these questions in the beginning.

[alexei]

Yes, it’s possible we weren’t ideal. On the other hand, sometimes you have to play the hand you’re dealt. I don’t think disagreement by itself is necesserily bad.

[Adam Zerner]

Reflection: we were also trying to build a two-sided marketplace. We needed writers, but writers wanted readers, but readers wanted good content. I think the correct way to solve that would have been to attract people with existing blogs /​ readership to switch to Arbital and bring their audience with them.

Yup. My first startup failed because I couldn’t crack that chicken-egg problem. FWIW, I think that http://​​platformed.info/​​ has some really solid content on the topic.

I recently finished reading Robin Hanson and Kevin Simler’s book The Elephant in the Brain, so I’m currently primed to always think, “maybe X isn’t really about Y”. In this case, I find myself thinking that bloggers/​writers are just interested in prestige, which is typically determined by the metrics of page views or users. As opposed to purely being interested in producing the best content possible.

If they truly are just interested in producing the best content possible, satisfied with the quality of their craftsmanship, and content with whatever users they happen to get, then yes, maybe Arbital would gain some writers and solve the chicken-egg problem that way. But if they just want prestige, well, I personally don’t see a path towards overcoming the chicken-egg problem.

Despite everything I just said… even if writers are solely after prestige… I still have a gut-level feeling that Arbital is on to something. If the UI truly allows writers to produce better content, when I take Paul Graham’s advice and try to live in the future, I find myself in a world where writers all use an Arbital-eque UI. The question of course is how to get to that future state, but I have a feeling that there is a path there.

[alexei]

Yes about the prestige. That was the realization I had in 2017.

Be careful: I think Arbital as an idea has evolved to be extremely sticky and “obviously good”. The antidote is to find a problem that actually exists and people want solved (and ideally will pay for). And only then take parts of Arbital that might provide a solution.

[JohnGreer]

Which specific content on http://​​platformed.info/​​ would you recommend?

[Adam Zerner]

It’s been a while since I spent time reading things there, and I don’t have particular things bookmarked. I just have memories of reading a lot of stuff there, liking it, and learning a lot from it. Sorry.

[JohnGreer]

That’s okay! Thanks for mentioning it at all. I’m exploring content on there now. :)

[Adam Zerner]

Furthermore, when we talked to a few rationalists directly, I just didn’t get the feeling of genuine helpfulness or enthusiasm. This was upsetting, because there aren’t that many big projects that the community does. So when I was doing Arbital, I guess I expected that more people would be on board, that more people would put in a bit of an extra effort to help us. But at best people put in minimal work (to satisfy us or themselves, I’m not sure). However, there was a limit to how upset I could be, because I very clearly recognized the same trait in me. So, while it’s still a sad state of affairs, I’d be a hypocrite for being upset with any particular person.

The reminds me of feelings I had during/​after my first startup. The startup intended to provide student reviews of colleges that address concrete, specific questions (rather than broad, open ended ones, which was the thing that currently existed). I too had the chicken-egg problem of writers/​reviewers not wanting to review for an empty audience, and readers not wanting to visit a site with no reviews.

At the time, I was a senior in college. I’m a friendly person, so I had lots of friends and acquaintences, both from college and from high school. Plan A was for them to think, “Oh, cool, Adam is starting a startup. Let me take 10-15 minutes to talk about my school a bit to help him out.” I figured that about half of the people I’m less close with would take a minimal amount of time to provide content, and then people who I was closer with would take more time to provide content.

My predictions were totally wrong. Lol. And I certainly felt some resentment. I felt that the cost to them was totally disproportionate to the benefit I’d receive, and that when you’re friendly with someone, it’s not nice to decline such a request.

I don’t mean to imply that your feelings are the same as mine, just that they remind me of what I felt.

Anyway, I sympathize with your feeling of being upset. And I also further update my beliefs away from expecting people to “help out” in situations like these.

[alexei]

Oh no, totally the same feelings. You get it. :)

However, since then I’ve gotten over that “should universe” and went back to “is universe”, where this is just how people are. Won’t be making that mistake twice. Sounds like we learned the same lesson. :)

[bfinn]

FWIW, and as you’ll no doubt have gathered, various of the issues you encountered are commonplace with software startups. But avoidable, particularly by involving or recruiting people with suitable business experience.

I note neither your post nor any comments seem to mention management or business experience as such. I don’t know if you or others involved had previous experience of running a business, but maybe little or none. The common startup model of figuring it out as you go along only works up to a point; things tend to descend into chaos sooner or later.

Managers are generally somewhat looked down on, particularly maybe by geeky types, who underestimate their importance. (Cf Eliezer recounted somewhere a meeting I think with a VC manager, who made some astute remark, and Eliezer was astonished to realize that there were very smart people in business. Why wouldn’t there be?) But business is no different from other skills (e.g. coding) - it isn’t straightforward, and years of experience is really valuable. You can’t just figure it out yourself or read up on it, you learn by doing. So involving or recruiting experienced managers (e.g. project managers, sales & marketing managers, indeed CEOs and CTOs) is very important. If it’s a small startup, maybe only one experienced businessperson is required early on, but it’s precarious with none.

[Adam Zerner]

I’ve clicked around Arbital a few times, but don’t feel like I have a great grasp of what it is. As I read through this post, I find myself thinking, “Gosh, I wish I really understood what Arbital is. If I did, I think I’d better understand and enjoy this post.” I can only provide one data point of perspective, but perhaps it would have been a good idea to start this post off describing what Arbital is before getting in to the timeline of how things went down.

[Adam Zerner]

Applause and thanks:

I would like to offer you some huge, huge applause for all of your self-honesty. I find it to be very refreshing, and inspiring. I’d also like to thank you for the educational value of this post.

Agreement:

None were successful, as far as I could tell. I didn’t see how Eliezer’s project was different, so I passed on it.

Reflection: Today, I’d probably try to sit down with Eliezer for longer and really try to understand what he is seeing that I’m not.

I think that’s a fantastic, and important lesson. It’s something I personally have been working on, and improving at as I get older and more mature. If someone I respect disagrees with me (in some form), I try to sound an alarm in my head saying, “STOP. If Person You Respect disagrees with you, that’s solid evidence in favor of you being wrong.”

Reflection: I think the rationality community can produce great insights, but mostly due to individual effort. There are great posts, but they rarely lead to prolonged conversations. And you very rarely see debates summarized for public consumption. (No wonder, it takes a lot of time and hard work!) There are a few counterexamples, but I think they prove the point by how much they stand out. (Best recent example I can think of is Jessica Taylor mediating a discussion between Eliezer and Paul Christiano and then writing it up.) (And, of course, not only do those things need to be written, but they also have to be read! And who has time to read…)

I very much agree. Personally, I feel hopeful and optimistic that we’ll get there.

The key difference from our path at the time was that instead of solving the Discussion problem and trying to get people to do new things, we’d simply focus on building a better tool for a thing people already do. Then once we had people on our platform, we could help improve the ongoing discussions.

I like that angle! Seems very reasonable.

Reflection: The decision process sounds a bit silly, but I don’t think it’s a bad one. I really prefer to do something decently useful, rather than sit around waiting for something perfect. I also still approve of the heuristic of accepting quests /​ projects from people you think are good at coming up with quests /​ projects. But if I did it again, I’d definitely put a lot more effort upfront to understand the entire vision before committing to it.

I also approve of this decision process. In particular, I applaud you for actually taking time to explore and consider a variety of ideas before picking one. I recall a Paul Graham essay where he talks about his belief that one of the biggest mistakes founders (applicants to YC?) make is picking the first idea they think of without taking a little time to ask if there is a better idea.

[Adam Zerner]

[Lawyers] Lesson 1: work only with a team who is recommended to you personally by someone you trust and who has worked with them before.

And what if you can’t meet those conditions (reasonably quickly)? I assume what you really mean is that meeting those conditions is really valuable, and (thus) that a recommendation from someone you trust but who hasn’t worked with them before isn’t nearly as valuable as it seems.

[alexei]

It’s too easy for people to just recommend their best lawyer friends. I suppose if you really trust your friends not to recommend their lawyer friends just because of the relationship (a big if!) then you could take their advice.

[kinrany]

Have you considered using Arbital as an API documentation platform?

[alexei]

Yes.

[Nowhere Man]

Thanks a lot for working on Arbital, and thanks for this writeup. It is always great to know how such projects went, why they stopped, etc…

Being one of the people having expressed an interest in an open source Arbital, I am trhilled to hear it might still happen!

[predictable agent]

Also from me: Thanks for writing this!

Could you explain how you had planned to make money with Arbital? (even if it is far-fetched)

And could you (or someone else) also expand on how to make money with Arbital 2.0 by putting it on a blockchain?

And, of course, put the entire thing on a blockchain to make crazy money in the meantime. ;)

(Sorry if that is obvious to many people, I don’t really know that crypto stuff that well.)

[Said Achmiz]

One thing that’s not clear to me:

Alexei, you mention that you were a single founder, i.e. Arbital was your startup.

What was Eliezer’s role, officially? Was he an investor? Or what?

[habryka]

My guess at Eliezer’s relationship with Arbital was similar to the relationship between a professor and one of their grad students, or the head of a large video game publisher and the head of an individual project. I.e. someone who doesn’t actively work on the project, but sets a large part of the tone and vision and has general veto right over how the project succeeds.

[Said Achmiz]

the relationship between a professor and one of their grad students, or the head of a large video game publisher and the head of an individual project

In both these cases, the first person is the formal superior of the second, and has the power to make decisions regarding the second’s continued employment (or lack thereof)—which authority is the source of their “veto right” over their subordinate’s project.

No such formal relationship existed here… did it? Again, my question concerns Eliezer’s official role in the project.

[habryka]

I can’t comment on the official relationship, so I won’t be of super much help here.

I think that for all intents and purposes Eliezer did indeed have the power to end the project if he wanted to, as well as to return the funds to investors. I don’t know the legal relationship between Eliezer and Arbital, though I don’t think that mattered particularly much. Though obviously Alexei should correct me if I am wrong here.

[alexei]

Advisor.

[Said Achmiz]

Is it common for an ‘advisor’ to have such pervasive and far-reaching control over what the founder of a startup does…?

[alexei]

No.