I’ve noticed that when I’ve been invited to view and comment on AI-safety related draft articles (in Google Docs), they tend to quickly attract a lot of extensive discussion, including from people who almost never participate on public forums like LessWrong or AI Alignment Forum. The number of comments is often an order of magnitude higher than a typical post on the Alignment Forum. (Some of these are just pointing out typos and the like, but there’s still a lot of substantial discussions.) This seems kind of wasteful because many of the comments do not end up being reflected in the final document so the ideas and arguments in them never end up being seen by the public (e.g., because the author disagrees with them, or doesn’t want to include them due to length). So I guess I have a number of related questions:
-
What is it about these Google Docs that makes people so willing to participate in discussing them?
-
Would the same level of discussion happen if the same draft authors were to present their drafts for discussion in public?
-
Is there a way to attract this kind of discussion/participants to public posts in general (i.e., not necessarily drafts)?
-
Is there some other way to prevent those ideas/arguments from “going to waste”?
-
I just remembered that LessWrong has a sharable drafts feature. (Where I think the initially private comments can be later made public?) Is anyone using this? If not, why?
Personally I much prefer to comment in public places, due to not wanting my comments to be “wasted”, so I’m having trouble understanding the psychology of people who seem to prefer the opposite.
This is a major issue the LessWrong team has thought about a lot. Building off ESRogs’ and Elo’s answers, the two main reasons are:
1) Privacy, comfort, and filtering.
2) Google Docs has great features and the best collaborative exobrain product.
Privacy/Comfort/Filtering
I think google docs ends up having the best overall conversation, but there is even a marked difference in feel between public LessWrong discussion and, say, Facebook discussion (holding participants constant).
Part of this is privacy concerns like ESRogs notes. If you represent an org maintaining a reputation, or you are discussing infohazardy material, or you have semi-secret information, you simply can’t chat freely in public – you have to think carefully about the ramifications of sharing your ideas. This makes sharing ideas harder, and sometimes unpleasant, so people end up doing it less. (semi-private venues like facebook, and many google docs, obviously carry at least some risk, but the risk feels a lot less pronounced).
But I think for a lot of documents, it’s less about literal secrecy or organizational PR, and more about an overall sense of safety/trust/filtering. In private you don’t have to worry as much about being wrong.
And, to some degree, “fear of being wrong” is something worth getting over. But there are practical matters that mean public discussion will always have some downsides.
In a public space, if you make an off-the-cuff remark that’s not fully defensible, you’ll often get people critiquing you who miss the point of what you’re actually trying to say, or (worse) are just actively trying to take you down a peg for social reasons. And you have to either spend a bunch of time arguing about something that wasn’t even the main idea you were trying to clarify or hash out, or not reply and look/feel like you’re ignoring a critic.
(And because of inferential distance, the critique might look reasonable to an onlooker, whereas you’ve already had this conversation 10 times and know the various arguments and counterarguments, and don’t feel like patiently explaining it again).
Abram’s recent post on intellectual cultures explores the details here. There’s a rough hierarchy of discussion that goes something like:
0. Open Verbal Combat
1. Face Culture.
2. Intellectual Debate
3. Mutual Curiosity
4. Exchanging Gears
Venues like LessWrong have a mix of people, some of whom are trying to do intellectual debate, some mutual curiosity, some exchanging gears (and occasional people doing pure verbal combat). Google docs lets you filter for people who are trying to do the kind of conversation you’re looking for.
Collaborative Exobrain Technology
Google docs is legitimately great. Important features include inline commenting, realtime collaboration, suggesting text, version history, etc.
These individual features are good, and they add up to a really powerful “group exobrain tool.” As of now Google Docs are simply much better than most other forum / blogpost technology for maximizing group-working-memory, and alternating between the “generation” step, “critique and discussion” step and the “distill” step of intellectual progress.
Some things that work better on google docs than many alternatives:
a. Peer Review.
The use case it sounded like you were most referring to. You can write up an initial paper exploring an idea. You invite smart people to comment on it. You might invite disparate groups of people at once, or you might take turns sending it to different groups of people who you think will do a good job of critiquing different aspects of the idea, or bring different perspectives.
The comments are attached to specific parts of the essay. In addition to immediately seeing the context of a given comment, I think there’s something like “humans using physical space to augment their working memory” (similar to memory palaces) where having a “location” for each comment helps people keep track of everything going on.
You can also give people the ability to suggest text, and then have people comment on those suggestions. So brainstorming and critique seamlessly flow back into the rewrite process. Whereas on a blog, if people comment, then the author needs to set aside time and activation energy to turn all those comments into a second draft, and then wait for further comments on whether that second draft actually addressed the issues.
(Plus, there’s something that feels “final” about posting something publicly, so doing a “second draft” feels a bit weird in the first place)
On google docs, the second draft happens sort of invisibly as you go along.
b. Planning.
Andrew Critch has been a huge proponent of using Google Docs to develop plans –if you have a course of action, be it a life plan, new project, or whatever, you can write it down.
Initially, writing it down forces you (at least somewhat) to notice what assumptions you’ve made, and make sure each step of the plan actually follows from the previous step.
Then, you show it to a couple people. At first it may not be very good so you just share it with one or two people as an overall sanity check. If you can find 4 people with different perspectives to vet the plan, and check assumptions you missed the first time, you can be a lot more confident that the plan makes sense.
The brainstorm/critique/distill/next-draft cycle applies to plans as well as idea-based-writing. But, in this case, the final product is not a paper to read – it’s a course of action that people can commit to. If a plan calls for multiple people to commit to a particular role, then part of the drafting process is to make sure each role makes sense… and to find people who would be willing to do it.
Just like a google-docs paper ends with you suddenly realizing that you have a finished draft, with little activation energy required to commit the final comments, ideally a google-docs plan ends with you realizing you have a viable strategy and a bunch of buy-in from people who helped shape it and are ready to start turning it into actual reality.
So what do we make of this?
I still think you’re right, that keeping all these comments hidden in drafts is wasting a lot of potential. I think there are potential solutions, but they require a fair bit of work, and accepting some of the constraints. People want the privacy and filtering while they’re hashing their ideas out.
The main thing lacking with google docs is the ability to turn, not just the finished writeup, but all the corresponding comments, into a publicly accessible work, with a click of a button.
(The LessWrong team is looking into some options for making LessWrong posts more google-docs-like, so that people and orgs can use it for internal documents while significantly lowering the barrier-to-entry to sharing them eventually, but it’s a big chunk of work and not something we can promise in the immediate future)
I would guess the main reasons are:
1. Privacy concerns—preferring not to go on-the-record with an off-the-cuff thought, especially if representing an organization.
2. The ability to comment on a specific line in a document, with the comment showing up in context.
Yeah, I really like how convenient that is.
I agree with both those points, and would add:
3. The fact that access to the doc is invite-only, and therefore people feel like they’ve been specifically asked to participate.
In a Google doc, a comment is perceived by both the author and the commenter as intending the be helpful (collaborative culture). In a lw post comments, there is often adversarial culture (you are wrong in ways xyz). It’s less fun to be around and it’s not as productive at encouraging people to participate.
If true, sounds like a bug and not a feature of lw.
I think it’s both somewhat of a bug that can be fixed (and I think we’ve made progress on this in the last year), and just a reality of public communication. I think there is a hierarchy of conversational modes that are more collaborative but require more and more trust among the participants (as Raemon mentioned in his answer), and that conversations usually end up at something close to the lowest level for which there is enough trust among all pairs of participants, which means that any large audience will usually imply a less collaborative communication style.
I think there are some ways around this, and I think the karma system is one of those (by increasing the weight more mutually trusted members have in the conversation, while not fully blocking out newcomers), but it’s overall a hard problem and it’s not clear to me that it can be solved completely with technical solutions.
I’m someone who both prefers and practises the ‘status quo’.
My impression is the key feature of this is limited (and author controlled) sharing. (There are other nifty features for things like gdocs—e.g. commenting ‘on a line’ - but this practice predates gdocs). The key benefits for ‘me as author’ are these:
1. I can target the best critics: I usually have a good idea of who is likely to help make my work better. If I broadcast, the mean quality of feedback almost certainly goes down.
2. I can leverage existing relationships: The implicit promise if I send out a draft to someone for feedback is I will engage with their criticism seriously (in contrast, there’s no obligation that I ‘should’ respond to every critical comment on a post I write). This both encourages them to do so, and may help further foster a collegial relationship going forward.
3. I can mess up privately: If what I write makes a critical (or embarrassing) mistake, or could be construed to say something objectionable, I’d prefer this be caught in private rather than my failing being on the public record for as long as there’s an internet archive (or someone inclined to take screen shots). (This community is no stranger to people—insiders or outsiders—publishing mordant criticisms of remarks made ‘off the cuff’ to infer serious faults in the speaker).
I also think the current status quo is a pretty good one from an ecosystem wide perspective too: I think there’s a useful division of labour between ‘early stage’ writings to be refined by a smaller network with lower stakes, and ‘final publications’ which the author implicitly offers an assurance (backed by their reputation) that the work is a valuable contribution to the epistemic commons.
For most work there is a ‘refining’ stage, which is better done by smaller pre-selected networks rather than of authors and critics mutually ‘shouting into the void’ (from the author’s side, there will likely be a fair amount of annoying/irrelevant/rubbish criticism; from a critic’s side, a fair risk your careful remarks will be ignored or brushed off).
Publication seems to be better for polished or refined work, as at this stage a) it hopefully it has fewer mistakes and so generally more valuable to the non-critical reader, b) if there is a key mistake/objection neglected (e.g. because the pre-selected network resulted in an echo chamber) disagreement between (‘steel-manned’) positions registered publicly and hashed seems a useful exercise. (I’m generally a fan of more ‘adversarial’ - or at least ‘adversarial-tolerant’ norms for public discussion for this reason.)
This isn’t perfect, although I don’t see the ‘comments going to waste’ issue as the greatest challenge (one can adapt one’s private comments to a public one to post, although I appreciate this is a costlier route than initially writing the public comment—ultimately, if one finds ones private feedback is repeatedly neglected, one can decline to provide it in the first place).
The biggest one I see is the risk of people who can benefit from a ‘selective high-quality feedback network’ (either contributing useful early stage criticism, having good early stage posts, or both) not being able to enter one. Yet so long as members of existing ones still ‘keep an eye out’ for posts and comments from ‘outsiders’, this does provide a means for such people to build up a reputation to be included in future (i.e. if Alice sees Bob make good remarks etc., she’s more interested in ‘running a draft by him’ next time, or to respond positively if Bob asks her to look something over).
This dynamic contributes to anxiety for me to comment in Google Docs, and makes it less fun than public commenting (apparently the opposite of many other people). I feel like if I fail to make a good contribution, or worse, make a dumb comment, I won’t be invited to future drafts and will end up missing a lot of good arguments, or entire articles because many drafts don’t get published until much later or ever. This is not a theoretical fear because I’ve “gotten in” to a couple of important and still unpublished drafts only by accidentally finding out about them and requesting invites.
Another thing that contributes to the anxiety is feeling a need to make a good first impression to people who are in the discussion who I’ve never talked to before because they don’t participate on public forums.
FYI, this was a significant update for me. I just wanted to note that this is a bigger part of my model now as opposed to an edge case tacked on.
I hadn’t actually been invited much to google docs where this dynamic would come up, but it makes sense that this experience would be common. (I’ve only actually talked to one other person who shared your experience, so still up for updating backwards again, but I don’t expect to)
Which should either cause me to downgrade the importance of the “feeling safer in a private space”, or have it apply a bit differently than I was expecting it to have applied. (I think it still applies to the author of the original paper, maybe less so to commenters. Although I do think think having a sense that your fellow commenters are being filtered for some kind of “on-the-same-page-ness” still improves the conversation in other ways)
FYI I’ve had this experience as well, though it’s not particularly strong or common.
Ah, yeah I think I’d feel the same if I were invited to docs that were either beyond my current reputation/social standing, or just, with a new group of people I don’t know as well. Especially if I considered it high stakes to keep getting into future similar discussions.
I think it’s a dramatically different experience when I invite or get invited to docs where I already know the people well, and already feel confident that I know how to contribute.
It would be neat if Google Docs had settings for their comments to only allow the doc’s authors do see your comments, or to make yourself anonymous from other commenters on the doc. I think these kinds of features could help reduce this kind of anxiety when reviewing people’s docs.
I think this underestimates the challenge. Empirically, people don’t crosspost those comments. Periodically saying “hey it’d be good if you crossposted those private comments” won’t change the underlying incentive structure.
(Similarly, the fact that one ‘could’ keep an eye out for posts and comments from outsiders won’t change the fact that people generally don’t)
Safety of draft-land. There’s no need to worry about the long term effects of being wrong in draft format. There is a reason to be worried about being wrong in published format.
(I definitely am guilty of discussing by Google doc. I have drafts unpublished in doc form right now)
May be we need a draft party? Like a meta-post on LW below which people will share their drafts? It would also help to coordinate who do what.
I’ve suggested these you mentioned in the past.
I’ve made mark up comments on posts.
I’ve tried to encourage/ask for a separate forum area for drafts.
I feel like these things were received poorly and discouraged so I don’t know how your proposal is different now.
I do think these are generally good ideas, but I think they need a combination of things going on at once in order to work well.
Google docs works for people who already have a network of peer reviewers, who already want to opt into that. I think your mark-up comments in google docs felt a bit out of the blue for people who weren’t planning to opt into it, so didn’t work as smoothly. (In google doc editing, you’re already in a draft mindset so revision-oriented comments feel more welcome)
For me there’s a huge difference between these two.
In gdocs I feel like it’s more okay to write “unpolished” comments. I think that’s mostly because the expectations are lower. Polishing my comments takes me 3-5x longer, which often takes away the motivation to comment at all.
In a public forum I worry more about provoking misleading impressions. For instance, in a gdoc shared with people who know me well, I’m not worried that a comment like “AIs might do [complex sequence of actions]” will get people to think that I have weirdly confident views about how the future might play out. In public conversations I’d experience a strong urge to qualify statements like that even though it feels tedious to do so.
A guess: google docs comments does not have karma system, so participants are free to tell what they want without worrying about losing points.