ARGH! OK, this seems like a really great idea, and I’m thrilled you did it. On the other hand, I feel like LessWrongers fail to optimize tons of effort by not collecting results when they do this class of experiment, so the people involved, and everyone else, fail to gain as much as they could. Obviously you won’t get a large sample, and the evidence is weak, but it’s REALLY valuable for you and for others considering running similar events.
If I were to specify what types of data I’d hope could be collected:
Ask mentees whether they feel they learned something interesting or worthwhile in each session. What was it, specifically? (Both regarding the topic, AND what they gained.) How useful was the session to them, on a scale from 1-10, where 1 is not worth their time, 3 is would consider doing it again, 7 is would certainly attend if it was done again, and 10 is would be willing to pay for similar events. Ask what they would want to change if it were done again.
Ask mentors whether they thought their mentee learned something interesting or worthwhile. Ask them to predict what their mentee would say on the above scale. Ask whether the mentors gained anything, and if so, what. Ask what they would want to change if it were done again.
These are all things you could do via a google survey, and you could tell the participants the results would be made public. You don’t need to have the datasets matched to figure out who mentored whom, but it might be kind of nice.
Maybe you could still do this, via an emailed survey? If you’d like me to set up the google form, I’d be thrilled to do so!
Yep, sounds right. Just didn’t think of it. I expect response rates to be really low to an emailed survey at this point and am not excited about doing it, unfortunately. If I had thought of it I would’ve had people fill out the survey at the end of the event.
ARGH! OK, this seems like a really great idea, and I’m thrilled you did it. On the other hand, I feel like LessWrongers fail to optimize tons of effort by not collecting results when they do this class of experiment, so the people involved, and everyone else, fail to gain as much as they could. Obviously you won’t get a large sample, and the evidence is weak, but it’s REALLY valuable for you and for others considering running similar events.
If I were to specify what types of data I’d hope could be collected:
Ask mentees whether they feel they learned something interesting or worthwhile in each session. What was it, specifically? (Both regarding the topic, AND what they gained.) How useful was the session to them, on a scale from 1-10, where 1 is not worth their time, 3 is would consider doing it again, 7 is would certainly attend if it was done again, and 10 is would be willing to pay for similar events. Ask what they would want to change if it were done again.
Ask mentors whether they thought their mentee learned something interesting or worthwhile. Ask them to predict what their mentee would say on the above scale. Ask whether the mentors gained anything, and if so, what. Ask what they would want to change if it were done again.
These are all things you could do via a google survey, and you could tell the participants the results would be made public. You don’t need to have the datasets matched to figure out who mentored whom, but it might be kind of nice.
Maybe you could still do this, via an emailed survey? If you’d like me to set up the google form, I’d be thrilled to do so!
Yep, sounds right. Just didn’t think of it. I expect response rates to be really low to an emailed survey at this point and am not excited about doing it, unfortunately. If I had thought of it I would’ve had people fill out the survey at the end of the event.