List of Q&A Assumptions and Uncertainties [LW2.0 internal document]

Context

1. This is the sec­ond in a se­ries of in­ter­nal LessWrong 2.0 team doc­u­ment we are shar­ing pub­li­cly (with min­i­mal edit­ing) in an effort to help keep the com­mu­nity up to date with what we’re think­ing about and work­ing on.

I sug­gest you first read this other doc­u­ment for con­text.

2. Caveat! This is in­ter­nal doc­u­ment and does not rep­re­sent any team con­sen­sus or con­clu­sions; it was writ­ten by me (Ruby) alone and ex­presses my own in-progress un­der­stand­ing and rea­son­ing. To the ex­tent that the mod­els/​ar­gu­ments of the other team mem­bers are in­cluded here, they’ve been filtered through me and aren’t nec­es­sar­ily cap­tured with high fidelity or strong en­dorse­ment. Since it was writ­ten on March 18th, it isn’t even up to date with my own think­ing
.

.

Epistemic sta­tus: Since the 18th when I first wrote this, I have many new lists and a lot more in­for­ma­tion. Yet this one still serves as a great in­tro into all the ques­tions to be asked about Q&A and what it can and should be.

Origi­nally writ­ten March 18, 2019

Re­lated: Q&A Re­view + Case for a Marketplace

  • HIGH LEVEL

    • Is it ac­tu­ally the case that Q&A for se­ri­ous re­search is this big, new, differ­ent thing which re­quires a big shift for peo­ple? Maybe it’s not such an ad­just­ment?

    • How will­ing are peo­ple to do se­ri­ous re­search work for oth­ers on the in­ter­net?

  • RESEARCH PROCESS (and suit­abil­ity for col­lab­o­ra­tion) <tease these out by talk­ing through their re­cent re­search>

    • Can “sig­nifi­cant re­search” be par­ti­tioned into dis­crete ques­tions?

      • Or is it more that there is a deeper big­ger ques­tion around which some­one needs to be­come an ex­pert, and that any ques­tion posed in down­stream of the real ques­tion and can’t be treated in iso­la­tion?

      • Per­haps talk to the Ought folk about this.

    • Do peo­ple have gen­eral open re­search ques­tions they want vaguely want an­swered and are will­ing to have sit unan­swered for a rel­a­tively long pe­riod of time?

      • Or do they mainly have (and pri­ori­tize) re­search ques­tions which are cur­rently part of their work?

    • How much in­ter­ac­tion be­tween the re­search re­quester and re­search con­trib­u­tor is re­quired?

      • Can some­one take a re­search ques­tion and ex­e­cute suc­cess­fully on their own with­out too much feed­back from the per­son re­quest­ing the re­search?

      • If nec­es­sary, does Q&A fa­cil­i­tate this ad­e­quately? Are back and forth com­ments good enough?

      • Are busy re­search re­questers will­ing to put in the time to in­ter­act with peo­ple try­ing to con­tribute, con­trib­u­tors who they don’t have know and haven’t nec­es­sar­ily vet­ted?

    • What kind of re­search ques­tions are amenable to the for­mat of LessWrong’s Q&A?

  • PERCEPTIONS AND PRIOR BELIEFS <should get an­swered semi-au­to­mat­i­cally in­ter­views>

    • Is the mix of re­search and less re­search-y ques­tions on Q&A now caus­ing peo­ple to not think of Q&A as a place for se­ri­ous re­search ques­tions?

    • What are peo­ple’s cur­rent im­pres­sions, ex­pec­ta­tions, an­ti­ci­pa­tions of LW’s Q&A, seg­mented by level of ex­po­sure?

      • e.g. if I tell some­one LessWrong has a Q&A with the goal of se­ri­ous re­search progress, what do they imag­ine? What’s their re­ac­tion?

      • Do peo­ple think that they could be helped by Q&A? Do they want to use it?

  • INCENTIVES, WILLINGNESS, & EXPERIENCE <get at these ques­tions by talk­ing through how in­ter­vie­wees might or might not use Q&A>

    • How much (and what kind) of in­cen­tives are needed for con­trib­u­tors to want to con­tribute?

      • Are boun­ties of cash prizes enough?

        • If yes, is it be­cause the money makes the effort worth it, OR

        • it just that cash prizes are a costly sig­nal is im­por­tant and once that’s clear, peo­ple would be glad to help?

        • Is bounty com­plex­ity an ac­tual is­sue?

        • Are peo­ple do­ing an EV calcu­la­tion with boun­ties such even if a nom­i­nal bounty is $500, peo­ple don’t nec­es­sar­ily think they’re worth a lot of work? Their EV is like $50

    • How good does the ROI need to be for ques­tion askers to want to use the plat­form?

    • How low does the time and at­ten­tion cost need to be for ques­tion askers to want to use the plat­form?

    • How much effort are ques­tion an­swer­ers will­ing to in­vest already?

      • It does look like that some Stack­Overflow ques­tions are very in­volved. So some peo­ple are will­ing to take time to an­swer things.

      • A few of the ques­tions/​an­swers on Q&A right now are pretty in­volved. Not many, but a few.

  • ADOPTION, POPULATION (TAM)

    • What is the pop­u­la­tion of ad­e­quately skil­led and available ques­tion an­swer­ers within the do­mains we care about? Is it enough to sup­port a good Q&A ecosys­tem?

      • How many peo­ple be­lieve they’re qual­ified? <prob­a­bly need more gen­eral pol­ling>

        • What’s the dis­tri­bu­tion of peo­ple in the 2x2 grid of “thinks they’re qual­ified” x”ac­tu­ally qual­ified”?

    • What user base of con­trib­u­tors do we have to reach be­fore the ques­tion asker ex­pe­rience is good enough to re­tain users?

  • OTHER <ex­pect to come up in talk­ing through their use of Q&A

    • Is pri­vacy a ma­jor is­sue for po­ten­tial ques­tion askers?

      • How do they feel if there are closed groups?

    • Is trust in re­search qual­ity an is­sue for ques­tion askers?

      • What does it take to eval­u­ate whether a re­search con­tri­bu­tion is good?

        • How much can it be done just by read­ing the con­tri­bu­tion or will it re­quire re­do­ing se­ri­ous work?

        • Are ques­tion askers will­ing to do this?

        • Are third par­ties will­ing to do the eval­u­a­tion?