Introducing Familiar, a quantified reasoning assistant (feedback sought!)

tl;dr: I’m mak­ing a thing that uses prob­a­bil­is­tic graph­i­cal mod­els to as­sist in draw­ing in­fer­ences from per­sonal data. You should check it out, and share with me your wis­dom/​user ex­pe­rience.

I had this not-com­pletely-origi­nal idea that there should be some kind of tool for eas­ily perform­ing statis­ti­cal in­fer­ence on Quan­tified Self-style data.

There are a lot of QS apps out there, but for the most part they seem to be de­signed for 1. a sin­gle do­main and/​or 2. record­ing things pri­mar­ily to com­bat akra­sia or (more of­ten) sat­ing cu­ri­os­ity/​as a lifestyle ac­ces­sory, rather than ac­tively helping you dis­cover cor­re­la­tions or de­ter­mine causal­ity be­tween things-you-do and things-you-care-about. Quan­tified Mind stands out as a coun­terex­am­ple, but I can’t come up with many oth­ers in that vein.

There are also com­mer­cial prod­ucts and pro­gram­ming lan­guages that al­low one to use ma­chine learn­ing to perform in­fer­ence on data, but they mostly seem to be pro­pri­etary and ex­pen­sive soft­ware aimed at busi­nesses, or free but in­tended to be used by sci­en­tists, en­g­ineers, etc.; noth­ing I’ve yet to find is re­ally suit­able for an in­di­vi­d­ual with­out a back­ground in statis­tics/​ma­chine learn­ing who just wants to learn what they can by smash­ing to­gether their Mood­scope and their FitBit.

In our era of FOSS, APIs, QS, and ML, this seems like a se­ri­ously lack­ing state of af­fairs. Hence, Fa­mil­iar.

Cur­rently, it con­sists of a com­mand line in­ter­face for stor­ing vari­able defi­ni­tions and data in a lo­cal database with­out too much fuss, build­ing a naive Bayes clas­sifier on those vari­ables, and find­ing max­i­mum like­li­hood es­ti­mates given the state of one vari­able for the states of all the other vari­ables. This is un­so­phis­ti­cated and not ex­tremely user-friendly, but those things will change in the near fu­ture. In the case where I keep work­ing on this for a very long time, I want to au­to­mate away as much record­ing as pos­si­ble (in­clud­ing things like mood and pro­duc­tivity), record ev­ery­thing with the high­est rea­son­able time re­s­olu­tion, plug into ev­ery other app out there that might provide use­ful data, use more com­plex ma­chine learn­ing al­gorithms to iden­tify causal­ity and gen­er­ate sug­ges­tions for per­sonal ex­per­i­men­ta­tion, and gen­er­ally have a piece of soft­ware that knows you so well it can help you think more like an ideal Bayesian rea­soner and thereby as­sist you in liv­ing your life (thus the name). Man­fred Macx’s glasses from Ac­celerando have some­thing like this in­side them, and I want it too.

Any­way, back to the pre­sent. You can help me by an­swer­ing whichever of these ques­tions ap­plies to you the most:

  • Would you use some­thing like this at all, or do you think the po­ten­tial for ex­tract­ing use­ful in­for­ma­tion out of messy per­sonal data is too low?

  • If you might use some­thing like this, but don’t want to use Fa­mil­iar in its cur­rent state, what do you think is the most im­por­tant fac­tor? e.g. “no GUI”, “not a web app”, “too man­ual”, “doesn’t con­nect to other stuff yet”, etc.

  • If you’re brave enough to start us­ing this now or even look at the source code, what mis­takes am I mak­ing? There are countless ways this could be eas­ier to use, more helpful, faster, more read­able, and oth­er­wise bet­ter, and you can tell me what those ways are.

Any­one’s feed­back will be ap­pre­ci­ated, but if you have ex­pe­rience in statis­tics or ma­chine learn­ing, Quan­tified Self stuff, writ­ing soft­ware that peo­ple ac­tu­ally use, or what­ever else might be rele­vant, I es­pe­cially want to hear your opinion. (Gw­ern gets to or­der me around.)
This is my main Hacker School pro­ject, by the way. If you like pro­gram­ming, I can’t recom­mend it enough, and ap­pli­ca­tions for the fall batch are cur­rently open.
(Aside: I don’t care much for the term “quan­tified self”. It’s ac­cu­rate in de­scribing what peo­ple have been do­ing with it so far, and I don’t re­ally ex­pect to get peo­ple to stop us­ing an already-pop­u­lar mostly-cor­rect la­bel, but I think there’s a lot of po­ten­tial in quan­tify­ing your in­ter­ac­tions with other peo­ple and your en­vi­ron­ment as well, and hav­ing the word “self” in the la­bel might un­duly limit imag­i­na­tions. “Quan­tified liv­ing” is closer to what I have in mind, but if you have catchier or more pre­cise sug­ges­tions I’d love to hear them too.)