I think it’s better to think about it on the question level:
There were a lot of questions (“What are you working on now”, for instance) that received basically the same answer every time, so they were useless.
Some other questions (“What do you need to get off your plate”) tended to receive the same answer every time, and they created an ugh field around the relevant thing by repeatedly reminding participants of their failure to do the thing.
Some questions were designed to catch rare-but-bad events (“Do you need to refill your medications”). I’m not sure if these were effective, because they didn’t exist for long enough to expect them to be answered positively. Bundling them with other Sphex question was a mistake, because they died along with it.
By far the most effective question (in terms of directly causing people to do things / change their behavior) was “Is there anything you can take care of right now? If so: do it”. Most other questions never did anything, this once got something accomplished ~1/week.
(Worth noting that we were only recording data towards the end of Sphex’s life, because that was when I started to organize it, and I care a lot about gathering data).
I’ve personally replaced Sphex with a set of check-ins spaced to occur at a reasonable rate (“When was the last time you made a git commit?”, “When was the last time you read a machine learning paper?”), etc. The general idea of “create check-ins” might work for other people, but the questions are probably too specific to me to be useful.
I like this post, and would like to see more posts like this.
Did you discover why Order of the Sphex failed?
I think it’s better to think about it on the question level:
There were a lot of questions (“What are you working on now”, for instance) that received basically the same answer every time, so they were useless.
Some other questions (“What do you need to get off your plate”) tended to receive the same answer every time, and they created an ugh field around the relevant thing by repeatedly reminding participants of their failure to do the thing.
Some questions were designed to catch rare-but-bad events (“Do you need to refill your medications”). I’m not sure if these were effective, because they didn’t exist for long enough to expect them to be answered positively. Bundling them with other Sphex question was a mistake, because they died along with it.
By far the most effective question (in terms of directly causing people to do things / change their behavior) was “Is there anything you can take care of right now? If so: do it”. Most other questions never did anything, this once got something accomplished ~1/week.
(Worth noting that we were only recording data towards the end of Sphex’s life, because that was when I started to organize it, and I care a lot about gathering data).
I’ve personally replaced Sphex with a set of check-ins spaced to occur at a reasonable rate (“When was the last time you made a git commit?”, “When was the last time you read a machine learning paper?”), etc. The general idea of “create check-ins” might work for other people, but the questions are probably too specific to me to be useful.