The Power to Judge Startup Ideas

This is Part III of the Speci­fic­ity Sequence

When Steve claims that Uber ex­ploits its drivers, he’s role-play­ing the sur­face be­hav­iors of an opinionated in­tel­lec­tual, but doesn’t bother to ac­tu­ally be an opinionated in­tel­lec­tual, which would re­quire him to nail down a co­her­ent opinion.

It turns out that a lot of star­tups are founded by peo­ple do­ing some­thing analo­gous to Steve: they role-play the sur­face be­hav­iors of run­ning a com­pany and build­ing a product, but don’t bother to nail down a co­her­ent pic­ture of what cus­tomers would ever come to their busi­ness for.

Startup Steves

Paul Gra­ham, cofounder of Y Com­bi­na­tor, calls this failure mode one of The 18 Mis­takes that Kill Star­tups:

Hav­ing No Spe­cific User In Mind
A sur­pris­ing num­ber of founders seem will­ing to as­sume that some­one, they’re not sure ex­actly who, will want what they’re build­ing. Do the founders want it? No, they’re not the tar­get mar­ket. Who is? Teenagers. Peo­ple in­ter­ested in lo­cal events (that one is a peren­nial tarpit). Or “busi­ness” users. What busi­ness users? Gas sta­tions? Movie stu­dios? Defense con­trac­tors?

I’m in the startup in­dus­try, and I watch a lot of star­tups com­mit­ting suicide by not be­ing spe­cific enough about who their cus­tomer is. From my per­spec­tive, the failure to be spe­cific isn’t just a top-18 mis­take, it’s the #1 mis­take that founders make.

If you watch Paul Gra­ham Office Hours at Startup School 2011, you can see for your­self that most of the founders on stage don’t seem to have a spe­cific idea of who they’re build­ing their product for and what differ­ence it makes in their lives. Eliezer ob­serves:

There was an ex­change in Paul Gra­ham [and Harj Tag­gar]’s office hours that went like this, while in­ter­view­ing a startup that did met­rics — an­a­lyz­ing pageviews, roughly — and the en­trepreneur was hav­ing great trou­ble de­scribing what they did that Mix­panel didn’t. It went on for a while. It was painful to watch.

Paul: I don’t get what the differ­ence is. I still don’t get what the differ­ence is. What’s the differ­ence be­tween you and Mix­panel?
En­trepreneur: The differ­ence is — when you have to sup­ple­ment — they’re a view com­pany and we’re a plat­form. That’s what it comes down to. They’re like a view, a re­port­ing com­pany. If you need some­thing they don’t have, a fea­ture -
Harj: So what’s an ex­am­ple of some­where you’d use your thing over Mix­panel? Can you give a use-case?
En­trepreneur: Yeah, I mean, we had rev­enue on day zero. There’s a good rea­son for um… it’s a start up, it’s a se­ries A com­pany in the daily deals space. One we’ve signed a so­cial game com­pany to -
Harj: And why do they pre­fer your thing?
Paul: That wasn’t what Harj was ask­ing.

The prob­lem (from the per­spec­tive of our pre­sent dis­cus­sion) is that the En­trepreneur did not un­der­stand that Paul and Harj were re­peat­edly ask­ing him to move down­ward on the lad­der of ab­strac­tion. When the En­trepreneur said “We had rev­enue on day zero”, he was try­ing to offer con­fir­ma­tion of the ab­stract state­ment “We can do things Mix­panel can’t”, but Paul and Harj still had no idea what his startup ac­tu­ally did.

How many early-stage star­tups have no spe­cific user in mind? I’d guess about 80% of them. And how bad is not hav­ing a spe­cific user in mind? So bad that I don’t think they should even be con­sid­ered a real startup, in the same way that Steve’s ar­gu­ment about Uber wasn’t a real ar­gu­ment.

Every Startup’s De­mol­ish­able Claim

Every startup founder makes the same claim to them­selves and to the in­vestors they pitch for fund­ing: “We’re go­ing to make a lot of money.” So what I do, nat­u­rally, is ask the founder to fur­nish a spe­cific ex­am­ple of that claim: a hy­po­thet­i­cal story about a sin­gle per­son who might be con­vinced to pay them a few bucks. And here’s how the con­ver­sa­tion usu­ally goes:

Founder: We’re go­ing to make billions of dol­lars and have mil­lions of users!
Liron: Ok, what’s a hy­po­thet­i­cal ex­am­ple of how you give one spe­cific user some value?
Founder: [Noth­ing]

Maybe they don’t liter­ally say noth­ing, but they say some­thing that doesn’t count for one of these rea­sons:

  • They an­swer in the ab­stract in­stead of giv­ing the ex­am­ple I re­quested of how they might give value to a spe­cific user

  • They choose a spe­cific ex­am­ple wherein their startup’s product or ser­vice isn’t any bet­ter for their hy­po­thet­i­cal user than the user’s available alternatives

At this point, I un­der­stand if you think I’m just knock­ing down a straw man, so here’s a real ex­am­ple.

Golden is a 2-year-old startup with $5M in fund­ing from An­dreessen Horow­itz, Founders Fund, and other no­table in­vestors. Their product is in­tended to be a su­pe­rior al­ter­na­tive to Wikipe­dia.

Here’s an ex­cerpt from the con­ver­sa­tion I had with Golden’s founder, Jude Gomila, on Twit­ter:

Liron: What spe­cific use case ex­ists on Golden to­day which is bet­ter than could have been achieved if the same amount of writer-effort had been spent on a pre-ex­ist­ing plat­form?
Jude: Quick tldr on this, some points cov­ered in the blog post, how­ever, 1. 1000x the topic space as a mis­sion 2. re­moval of no­ta­bil­ity req 3. Us­ing AI to au­to­mate flows 4. Us­ing AI to com­pile knowl­edge 5. Bet­ter fact val­i­da­tion/​hi res cites 6. Bet­ter schema eg timeline 7. Fea­tures and func­tions eg fav­ing, ac­tivity feed, par­allel rab­bit hole 8. query re­sults like these as well plus many many more. Have you tested the ed­i­tor: magic cells, cita­tions product and AI sug­ges­tions?

As for the spe­cific ex­am­ples Jude pro­vided in re­sponse to my ques­tion… well, I’ll just give you the first two and you can judge them for your­self:

1. https://​​​​wiki/​​Cry­obac­terium vs https://​​en.wikipe­​​wiki/​​Cryobacterium

2. https://​​​​wiki/​​Ginkgo_Bioworks vs https://​​en.wikipe­​​wiki/​​Ginkgo_Bioworks

Golden has re­ceived more fund­ing than star­tups nor­mally get be­fore hav­ing any mar­ket trac­tion to show, and the com­pany’s high pro­file makes it a juicy ex­am­ple to illus­trate my point here. But there are countless other com­pa­nies I could have sin­gled out in­stead. Re­mem­ber, the ma­jor­ity of early-stage star­tups are op­er­at­ing in this same failure mode. There are enough ex­am­ples of star­tups visi­bly failing this way that I’ve started a blog to col­lect them.

The Value Prop Story Test

When I chat with a founder about their new startup, or I look through the slide deck that they’re us­ing to pitch their idea to in­vestors, the first thing I do is try to pull out what I call a Value Prop Story: one spe­cific story wherein their startup gives some­body some value.

A well-formed Value Prop Story must fit into this tem­plate:

  1. De­scribe a spe­cific per­son with a spe­cific problem

  2. De­scribe their cur­rent best effort to solve their problem

  3. De­scribe why it’s still a problem

  4. De­scribe how their life gets bet­ter thanks to you

I’ve pre­vi­ously ob­served that tel­ling a well-formed Value Prop Story doesn’t re­quire you to show any mar­ket re­search or em­piri­cal ev­i­dence val­i­dat­ing the qual­ity of your idea. This is like how Steve didn’t yet need to give us any em­piri­cal or the­o­ret­i­cal jus­tifi­ca­tions for his claim about Uber’s driver ex­ploita­tion, he just needed to tell us a story about one hy­po­thet­i­cal spe­cific driver get­ting ex­ploited in a spe­cific way.

Who is a spe­cific hy­po­thet­i­cal per­son who will use your product, and in which spe­cific sce­nario will they use it? That’s it, that’s the ques­tion most star­tups can’t an­swer.

An­swer­ing this ques­tion seems ob­jec­tively easy to me, in the sense that a well-de­signed AI wouldn’t stum­ble over it at all. What about for a brain though, is it a tough men­tal op­er­a­tion?

Ac­tu­ally, I think you’ll find that this is an easy men­tal op­er­a­tion if you ac­tu­ally have a good startup idea. Here’s a Value Prop Story I wrote about my own startup with­out much trou­ble:

  1. De­scribe a spe­cific per­son with a spe­cific prob­lem
    23 year old male who can’t get a date

  2. De­scribe their cur­rent best effort to solve their prob­lem
    He gets a Tin­der ac­count and does his best to use it on his own

  3. De­scribe why it’s still a prob­lem
    His matches barely re­spond to his mes­sages, and when they do, the con­ver­sa­tion feels bor­ing and forced. He uses it for 1 hour ev­ery day but only man­ages to get 1 date ev­ery 2 months.

  4. De­scribe how their life gets bet­ter thanks to you
    Once Re­la­tion­ship Hero coaches guide him through writ­ing his texts, he sud­denly has much bet­ter con­ver­sa­tions that re­sult in a date each week

Since my startup ac­tu­ally has a broad range of use cases (clients come to us for help with a broad range of re­la­tion­ship is­sues), this Value Prop Story isn’t par­tic­u­larly rep­re­sen­ta­tive of what we do. Its job was merely to prove that there are more than zero plau­si­ble spe­cific use cases for Re­la­tion­ship Hero, and it gets that job done.

Given how easy this ex­er­cise is—we’re talk­ing five min­utes, tops—I find it mind-bog­gling that 80% of star­tups reck­lessly skip it and go straight to, um… what­ever else they think star­tups are sup­posed to do. Paul Gra­ham writes:

Another of the char­ac­ter­is­tic mis­takes of young founders is to go through the mo­tions of start­ing a startup. They make up some plau­si­ble-sound­ing idea, raise money at a good val­u­a­tion, rent a cool office, hire a bunch of peo­ple. From the out­side that seems like what star­tups do. But the next step af­ter rent a cool office and hire a bunch of peo­ple is: grad­u­ally re­al­ize how com­pletely fucked they are, be­cause while imi­tat­ing all the out­ward forms of a startup they have ne­glected the one thing that’s ac­tu­ally es­sen­tial: mak­ing some­thing peo­ple want.

Why would you spend time and money build­ing a product when you can’t yet tell a spe­cific Value Prop Story? I think it’s be­cause de­sign­ing and build­ing a product is fun and gives you a false sense of con­trol. You can lie to your­self the whole time about the like­li­hood that you’ll even­tu­ally get peo­ple to use what you’re build­ing.

But peo­ple usu­ally won’t use what you’re build­ing. When­ever a new startup ex­cit­edly launches their product for the first time, the most likely out­come is that they get liter­ally zero users.

The Se­cret fa­mously claimed that wish­ing for some­thing makes the uni­verse give it to you, which is BS, but the con­verse is true: If you haven’t made a spe­cific enough wish about what your ini­tial mar­ket trac­tion is sup­posed to look like, then the uni­verse won’t give you any trac­tion.

The Ex­tra-Pow­er­ful San­ity Check

Is it healthy for us to be ob­sessed with judg­ing star­tups and de­mol­ish­ing claims about their value propo­si­tions? When we say that a startup idea is bad on ac­count of lack­ing a Value Prop Story, is it right and proper to feel pleased with our­selves, or are we be­ing gra­tu­itously ad­ver­sar­ial?

Along these lines, Mix­panel cofounder Suhail Doshi has tweeted:

I get lit­tle satis­fac­tion stomp­ing on some­one’s startup idea. It’s so easy to. Some­where deep, hid­den in their ab­stract de­scrip­tion is a dis­tinct yet nar­row prob­lem worth solv­ing that’s sig­nifi­cant. It’s more fun to at­tempt find­ing it, to­gether.

I ba­si­cally agree with this, and I ba­si­cally agree with the com­menters on my de­mol­ish bad ar­gu­ments post who em­pha­sized that we should seek to shine a light on what­ever ker­nels of truth our con­ver­sa­tion part­ner may have brought to the table.


Have you ever san­ity checked some­thing?

A san­ity check is like when you punch 583x772 into your calcu­la­tor, and you quickly mul­ti­ply the two right­most digits in your head, 3 x 2 = 6, and then con­firm that the calcu­la­tor’s out­put ends in a 6. If you ever ac­ci­den­tally punch the wrong se­quence of keys into the calcu­la­tor, then you’ll be pretty likely to see the calcu­la­tor’s an­swer end in some­thing other than a 6. It’s a good use of two sec­onds of your time to calcu­late 3 x 2; you get a sub­stan­tial dose of Bayesian ev­i­dence for your trou­ble.

The Value Prop Story test is like­wise a san­ity check for startup ideas. In the­ory, of course a startup founder who is already hiring a team of en­g­ineers and build­ing a soft­ware product should be able to de­scribe how one spe­cific user will get value from that product. In prac­tice, they of­ten can’t. And it’s easy for us to quickly check.

Here’s what’s crazy though: We usu­ally ex­pect san­ity checks to have a low rate of de­tect­ing failures. You ex­pect to suc­cess­fully mul­ti­ply num­bers on your calcu­la­tor most of the time, but you do the 3 x 2 san­ity check any­way be­cause it’s quick. But with the Value Prop Story test, you’ll see a high rate of failures!

A san­ity check with a high failure rate is a rare treat; it’s an ex­tra-pow­er­ful san­ity check. When you’re lucky enough to have an ex­tra-pow­er­ful san­ity check in your toolbox, don’t make it a fi­nal step in your pro­cess, make it the first step in the pro­cess.

So here’s how you can use the Value Prop Story test to up­end the tra­di­tional or­der of op­er­a­tions for build­ing a startup: First, re­peat­edly san­ity check your­self with the Value Prop Story test un­til you pass it. Se­cond, do ev­ery­thing else. In all se­ri­ous­ness, I’ve recom­mended that early-stage startup founders fol­low this flow chart:

But how should we treat founders who are stuck in the flowchart’s “Give Value to One Per­son” stage?

When some­one is strug­gling to pass a san­ity check, that doesn’t mean we should write off their po­ten­tial to suc­ceed. It means we should fo­cus our effort on helping them pass the san­ity check.

Ap­ply­ing the Value Prop Story test is like plac­ing a low bar in a founder’s path. Yes, the bar will trip the ones who aren’t see­ing it. But for the ones who do see the bar, they can step up onto it and then be on their way. And the next step in their path, such as build­ing a qual­ity product, or build­ing a sales fun­nel, is sure to be a steeper one than that lit­tle first one.

Next post: The Power to Make Scien­tific Breakthroughs

Com­pan­ion post: Ex­am­ples of Examples