[Question] What is required to run a psychology study?

I pe­ri­od­i­cally find my­self wish­ing some­one had run an ex­per­i­ment on a par­tic­u­lar topic. But they haven’t.

Often, it seems like there’s a rel­a­tively easy ex­per­i­ment you can run, which would give me some ev­i­dence about it. Maybe it wouldn’t be perfect ev­i­dence, but it would be bet­ter than me mak­ing stuff up from my arm­chair.

The two clusters of things-I-can-imag­ine-be­ing-quite-hard are:

  • Le­gal requirements

  • Ac­tu­ally do­ing the sci­ence right (avoid­ing sam­pling bi­ases, ac­tu­ally test­ing the right vari­able, etc)

Le­gal Stuff

Scott’s IRB Night­mare sug­gests that the le­gal re­quire­ments can be quite awful. I’m un­clear on what those re­quire­ments are if I don’t want to pub­lish any­where, don’t plan to in­ter­face with any hos­pi­tal bu­reau­cra­cies, I just want to ask a bunch of peo­ple to try some­thing and see how it goes (and maybe make a LessWrong blog­post so peo­ple on LessWrong can know too)

Am I al­lowed to just go out and ask a whole bunch of peo­ple stuff? If I want to know things about 8 year olds or 16 year olds, if their par­ents sign a form and I’m not do­ing any­thing es­pe­cially weird or trau­matic, would that be fine or would ter­rible things hap­pen to me? (Could I repli­cate the Marsh­mal­low Test?)

Do­ing Science Good Enough

I have a sense that there’s a lot of ways to de­ceive your­self if you have a pet psy­chol­ogy the­ory. But… I dunno it also seems like LessWrong col­lec­tively should be pretty good at this. The thing I might want to do (or get oth­ers to do some­times) is post a plan for a study here, and get cri­tiqued un­til it seems like an ac­tu­ally good plan, do the plan, write up the re­sults.

Par­tic­u­lar Things I was in­ter­ested in:

(not meant to be ex­haus­tive, im­por­tant, or par­tic­u­larly achiev­able plans, just the things that gen­er­ated this ques­tion)

  • How do most peo­ple re­late to their job? (There’s a sur­vey study that asks if peo­ple con­sider it a job, a ca­reer, or a call­ing, but the study had a very nar­row range of par­ti­ci­pants – peo­ple who worked at one par­tic­u­lar place, and I’m cu­ri­ous how a wider va­ri­ety of peo­ple would re­spond)

  • How do most peo­ple re­late to am­bi­tion? (where by am­bi­tion I mean “form plans to cre­ate or change things that will im­pact large num­bers of peo­ple.”)

  • How many 8 year olds can learn to pro­gram? Can they im­ple­ment Fiz­zBuzz? How many 16 year olds? How many 30 year olds? I’ve heard that peo­ple ei­ther have-or-don’t-have a “pro­gram­mer trait” that’s hard to learn, and I’m not sure how much of that is true at all, and if so if it’s more about na­ture or nur­ture.