‘Temporary MAP stance’ or ‘subjective probability matching’
Temporary MAP stance or subjective probability matching are my words for useful mental manoeuvres for research, especially when dealing with confusing or prepradigmatic or otherwise non-crisp domains.
MAP is Maximum A Posteriori i.e. your best guess after considering evidence. Probability matching is making actions/guesses proportional to your estimate of them being right (rather than picking the single MAP choice).
By this manoeuvre I’m gesturing at a kind of behaviour where you are quite unsure about what’s best (e.g. ‘should I work on interpretability or demystifying deception?’) and rather than allowing that to result in analysis paralysis, you temporarily collapse some uncertainty and make some concrete assumptions to get moving in one or other direction. Hopefully in so doing you a) make a contribution and b) grow your skills and collect new evidence to make better decisions/contributions next time.
It happens to correspond somewhat to a decent heuristic called Thompson Sampling, which is optimal under some conditions for some uncertain-duration sequential decision problems.
HT Evan Hubinger for articulating his take on this in discussions about research, and I’m certain I’ve read others discussing similar principles on LW or EAF but I don’t have references to hand.
‘Temporary MAP stance’ or ‘subjective probability matching’
Temporary MAP stance or subjective probability matching are my words for useful mental manoeuvres for research, especially when dealing with confusing or prepradigmatic or otherwise non-crisp domains.
MAP is Maximum A Posteriori i.e. your best guess after considering evidence. Probability matching is making actions/guesses proportional to your estimate of them being right (rather than picking the single MAP choice).
By this manoeuvre I’m gesturing at a kind of behaviour where you are quite unsure about what’s best (e.g. ‘should I work on interpretability or demystifying deception?’) and rather than allowing that to result in analysis paralysis, you temporarily collapse some uncertainty and make some concrete assumptions to get moving in one or other direction. Hopefully in so doing you a) make a contribution and b) grow your skills and collect new evidence to make better decisions/contributions next time.
It happens to correspond somewhat to a decent heuristic called Thompson Sampling, which is optimal under some conditions for some uncertain-duration sequential decision problems.
HT Evan Hubinger for articulating his take on this in discussions about research, and I’m certain I’ve read others discussing similar principles on LW or EAF but I don’t have references to hand.