I actually participated in the original relay experiment described here as one of the players, and this seems like a good place to share my experience with it.
The format of the iterative collaboration with different people felt a lot like collaborating with myself across time, when I might drop a problem to think about something else and then come back to it. If I haven’t taken the right notes on what I tried and (crucially) what I should do next, I have to pay a tax in time and mental resources to figure out what I did.
It was interesting to experience it in this case, as the combination of the time limit and collaboration with other people meant that I felt a lot of pressure each time figuring out what happens.
More concretely, for some problems, I could actually understand the problem quickly enough and follow along with the current solution so far. For others, I would be greeted with 10 page google doc, documenting all the things that were tried and without a clear next step. In a few cases, it took me 10 minutes just to read through the full problem and things that were tried and I didn’t even have time to rephrase it.
The cases where I felt I made the most progress on the problem, there was a clear next step (or list of possible steps) that I could implement. For instance, for one of the prime problems a clear intermediate step was that we needed a fast primality test, and that was a feasible 10 min implementation that didn’t need understanding of the full problem.
I still think a lot about my experiment in the Relay, as it has affected how I think about documenting my progress on my projects so that I can take them up faster next time.
I agree that at first glance, it may seem like advertising, but it is different in quite a few ways:
The service Jonah and Vipul offer is free. They are doing this for the benefit of the community, not for themselves.
Jonah and Vipul seem well qualified to actually advise people.
This is in the spirit of self-study and improvement that perpetuates Less Wrong.
Really, I see nothing wrong with offering rational advising on a site that aims to improve human rationality.