My guide to lifelogging

I’ve defended the prac­tice of lifel­og­ging as a means of life ex­ten­sion here. In this post I’ll provide a fairly com­pre­hen­sive guide on how to lifelog. Since lifel­og­ging ex­ists on a spec­trum from “tak­ing a pic­ture ev­ery so of­ten” to “record­ing ev­ery sin­gle de­tail of your life, in un­com­pressed HD video along with con­tin­u­ous MRI scans and stor­age in a nu­clear-safe vault” this guide will pre­sent two cat­e­gories for lifel­og­ging, the first for lower cost op­tions and the sec­ond higher-cost op­tions. “Cost” here refers not only to the mon­e­tary price of buy­ing the equip­ment, but also the con­ve­nience costs of set­ting up the equip­ment, stor­ing the data, and per­haps so­cial em­bar­rass­ment.

Over the last sev­eral months I have spent many hours of re­search to de­ter­mine the best se­tups in terms of time and en­ergy re­quired to record my life. I also recom­mend view­ing Mati Roy’s setup.

I in­tend to up­date this guide as I learn more, so keep in mind that this post is a work-in-progress.

Lower-cost lifelogging

Archiv­ing so­cial media

The low­est hang­ing fruit of lifel­og­ging is prob­a­bly cre­at­ing a long-term archive of your so­cial me­dia data. The method of archiv­ing your so­cial me­dia data will nec­es­sar­ily de­pend on the web­sites you visit, but here are some guides for com­mon web­sites:

Keylogging

I do not key­log my­self, but Mati Roy has in­formed me that Spyrix works well.

Tak­ing pictures

Th­ese days, smart­phones gen­er­ally have high qual­ity cam­eras, and are much less of a has­sle than buy­ing pro­fes­sional equip­ment.

Storage

Since ev­ery­thing in the lower-cost sec­tion here takes up a small amount of space, cloud stor­age is an ap­pro­pri­ate way of stor­ing data for the long-term. Google Drive offers 15 GB of free stor­age, though I would also sug­gest stor­ing a lo­cal copy of ev­ery file along with check­ing out the sub­header on au­dio and video com­pres­sion, and the sub­header on long-term stor­age in the sec­tion on higher cost lifel­og­ging.

[ETA: Mati Roy com­ments be­low that Google Pho­tos offers un­limited free stor­age for pic­tures and video.]

Higher-cost lifelogging

Screen recording

The most salient way that I lifelog is by record­ing ev­ery­thing that hap­pens on my com­puter screen, along with a full video of my face and room. I achieve this setup by us­ing Open Broad­caster Soft­ware (OBS) and record a con­tin­u­ous split-screen be­tween my screen and my USB cam­era.

I have heard that OBS can be quite an­noy­ing to use if you use a lap­top, as it turns up the fans and gen­er­ally uses too much of the CPU. There­fore, I recom­mend build­ing a com­puter with a high qual­ity CPU and buy­ing a wide-an­gle USB cam­era, along with a micro­phone to record.

The desk­top com­puter I use is five years old, so I can­not recom­mend the ex­act parts I bought at the time. I also do not recom­mend us­ing the USB cam­era that I bought, as it does not have a wide enough an­gle for my tastes. In­stead, I recom­mend brows­ing the sub­red­dits /​r/​buil­dapc and /​r/​buil­dapcforme un­til you have a de­cent idea of what goes into build­ing a com­puter.

I would es­ti­mate the min­i­mum cost of a desk­top com­puter that can re­li­ably run OBS with­out prob­lems at around $500, if you know what you are do­ing. But a price tag of at least $750 may be bet­ter if you don’t want to run into is­sues later. This bench­mark­ing site, and this one are use­ful for de­ter­min­ing low-cost high-qual­ity CPUs. At the mo­ment I sug­gest get­ting around a $200 to $300 newest gen­er­a­tion AMD Ryzen CPU.

Night­time recording

The value of record­ing your­self sleep is ar­guable, so I do not sug­gest this to ev­ery­one. My own jus­tifi­ca­tion was to have a sense of com­plete­ness in my lifel­og­ging, and feel like I wasn’t ever miss­ing a mo­ment.

That said, I pur­chased this USB in­frared cam­era to record my­self at night, and it works well. It also func­tions as a day-time cam­era, au­to­mat­i­cally switch­ing to in­frared when the lights go out, mak­ing it suit­able as an all-day record­ing cam­era. I also pur­chased this fit­ness watch to track my sleep, though this as­pect is ob­vi­ously not nec­es­sary.

Just as in the above sec­tion, I use OBS to fa­cil­i­tate the record­ing. It’s worth un­der­stand­ing how pro­files and scene col­lec­tions work in OBS so that you can sim­plify your setup.

When I’m not at my desk­top

Record­ing at my desk­top is nice, since I can use OBS, but when I’m on-the-go I have two main ways of record­ing, us­ing au­dio and video.

Audio

The first method is au­dio record­ing us­ing my phone. I have an iPhone at the mo­ment, and there­fore I recom­mend An­droid users to look at Mati Roy’s ad­vice. I pur­chased this om­ni­di­rec­tional lava­lier micro­phone along with this light­ning-to-head­phone jack con­nec­tor, and am gen­er­ally pleased with the qual­ity.

I use the app Dic­ta­phone, but I’m not con­fi­dent at all that this app is the best. It was sim­ply the first thing I looked at for iOS.

The lava­lier micro­phone con­nects to my shirt, sort of like in this pic­ture, and I don’t gen­er­ally have to think about it much when I’m on the go. Of course, I urge po­ten­tial lifel­og­gers to make sure that they have the con­sent of all par­ties be­fore record­ing peo­ple on-the-go.

In or­der to save phone bat­tery, I also pur­chased this voice recorder, which has sur­pris­ingly long bat­tery life and ac­cept­able stor­age. How­ever, I mostly don’t use the voice recorder any­more since I have switched to mainly record­ing video while I’m on-the-go, as I ex­plain in the next sec­tion.

Video

If you aren’t satis­fied with record­ing au­dio con­tin­u­ously on-the-go, you can switch to us­ing video. I ex­per­i­mented with pur­chas­ing an ac­tion cam­era (ie. what Go Pros are) for this pur­pose, but then soon re­al­ized that there was a bet­ter al­ter­na­tive.

I now recom­mend lifel­og­gers pur­chase a body cam­era, of the type used by po­lice. Here some of the pros and cons of body cams com­pared to ac­tion cam­eras:

Pros:

  • They gen­er­ally have much longer bat­tery life (very im­por­tant)

  • Most have na­tive in­frared record­ing so you can record at night

  • Body cams are built to al­low you to eas­ily clip it onto cloth­ing (this makes con­tin­u­ous record­ing less awk­ward)

  • They tol­er­ate shock dam­age, such as drop­ping the cam­era, more than many ac­tion cameras

Cons:

  • The video qual­ity is lower

  • Fewer fea­tures are available

  • There are fewer on­line re­sources for op­er­at­ing body cams

After sub­stan­tial re­search I de­cided to buy this body cam­era. The pri­mary rea­son I went with it over other cam­eras was be­cause it had a de­tach­able bat­tery (with an ex­tra), and de­tach­able stor­age (but you must pur­chase the SD card on your own). The main down­side is that the lens an­gle is only 140 de­grees com­pared to 170 in some other body cams.

The body cam is well-built and is much lighter in weight than you might ex­pect. It con­nects eas­ily to my com­puter via a USB ca­ble that en­ables me to trans­fer the video files to long-term stor­age.

To min­i­mize stor­age costs, I record in 480p and com­press all my files once I have trans­ferred them to my com­puter (see next sec­tion). The body cam al­lows an op­tion for on-board stor­age dur­ing record­ing, but I don’t use it be­cause it seems to work by sim­ply halv­ing the bi­trate of the video with­out any­thing in­tel­li­gent in­volved. A similar thing seems to hap­pens when you turn on the op­tion for pre-record­ing for some rea­son.

On a full charge I can get over 5 hours for each bat­tery, and it is easy to re­place the bat­tery when the body cam dies. With a 128 GB SD card it can hold about 68 hours of con­tin­u­ous 480p qual­ity video be­fore it runs out of space.

I have tried var­i­ous ways of con­nect­ing it to my body, and the thing that seems to work the best is sim­ply con­nect­ing the body cam to my pants, or belt, as shown in the image be­low. Un­for­tu­nately, you do have to tuck in your shirt or else the body cam won’t be able to see much. On the bright side, this means that if you want to hide that you are wear­ing a body cam, all you have to do is make sure your shirt cov­ers it.

More down­sides of keep­ing it on your pants in­clude the fact that it doesn’t record peo­ple’s faces if you are talk­ing to some­one close by, and it doesn’t record very well when you are sit­ting down at a table or sit­ting more gen­er­ally.

See this Google Drive video for a sam­ple of the post-com­pres­sion qual­ity us­ing the body cam. I think that Google Drive com­presses video up­loaded there, so make sure to down­load it to see the real qual­ity, as it’s only 21.9 MB.

Au­dio and video compression

Video takes up a LOT of stor­age un­less you com­press. Au­dio is similar, though less ex­treme. There­fore, be­fore trans­fer­ring my files into long-term stor­age, I always com­press them into some­thing of ac­cept­able size.

I use FFm­peg to com­press my me­dia files, which works well on Ubuntu, but I have not tried it out on other op­er­at­ing sys­tems. To com­press my videos I run this bash script,

#!/usr/bin/env bash

for i in *.MOV;
do name=`echo "$i" | cut -d'.' -f1`
echo "$name"
ffmpeg -i "$i" -c:v libx264 -preset veryslow -crf 24 -strict -2 "${name}.mp4"
done

rm *.MOV

The two most im­por­tant things to un­der­stand about the script above are the op­tions

-pre­set veryslow

and

-crf 24

Th­ese op­tions de­ter­mine the qual­ity and size of the video. I recom­mend choos­ing qual­ity and size de­pend­ing on your own tol­er­ance for stor­age costs (see the sec­tion on long-term stor­age be­low). The the FFm­peg doc­u­men­ta­tion ex­plains these op­tions in more de­tail,

A pre­set is a col­lec­tion of op­tions that will provide a cer­tain en­cod­ing speed to com­pres­sion ra­tio. A slower pre­set will provide bet­ter com­pres­sion (com­pres­sion is qual­ity per file­size). This means that, for ex­am­ple, if you tar­get a cer­tain file size or con­stant bit rate, you will achieve bet­ter qual­ity with a slower pre­set. Similarly, for con­stant qual­ity en­cod­ing, you will sim­ply save bi­trate by choos­ing a slower pre­set.
Use the slow­est pre­set that you have pa­tience for. The available pre­sets in de­scend­ing or­der of speed are ul­tra­fast, su­per­fast, very­fast, faster, fast, medium – de­fault pre­set, slow, slower, verys­low.
[...]
The range of the CRF scale is 0–51, where 0 is lossless, 23 is the de­fault, and 51 is worst qual­ity pos­si­ble. A lower value gen­er­ally leads to higher qual­ity, and a sub­jec­tively sane range is 17–28. Con­sider 17 or 18 to be vi­su­ally lossless or nearly so; it should look the same or nearly the same as the in­put but it isn’t tech­ni­cally lossless.
The range is ex­po­nen­tial, so in­creas­ing the CRF value +6 re­sults in roughly half the bi­trate /​ file size, while −6 leads to roughly twice the bi­trate.

For au­dio, I use this com­mand,

find -name "*.WAV" -exec ffmpeg -i {} -acodec  libmp3lame -qscale:a 5 -ab 128k {}.mp3 \;

Long-term storage

For large amounts of short-term stor­age, you can visit the web­site dis­kprices.com to view the cheap­est stor­age available to con­sumers. Per­son­ally, I recom­mend get­ting SSD stor­age as op­posed to HDD stor­age for short-term use, as even though it is more ex­pen­sive, it is also much faster.

How­ever, since both SSDs and HDDs are not built to store data for decades with­out cor­rup­tion, the best op­tion at the mo­ment is likely burn­ing data onto blu ray discs. You can find cases of 50 blu ray discs that hold 22.5 GB for be­tween $20 to $25. How­ever, the real costs of long-term stor­age will be higher than this for two rea­sons,

  • For the long-term, ideally you should keep at least two copies of ev­ery file, and you should store them in sep­a­rate lo­ca­tions.

  • Data burn­ing of­ten fails with a rate of be­tween 10 to 20 per­cent, which means that your true cost es­ti­mates should take into ac­count the fact that many discs will be use­less.

I pur­chased this blu-ray burner, which works ac­cept­ably but I’m un­sure whether it is the best op­tion. I also pur­chased a few of these cases which can hold a lot of discs quite cheaply.

I sug­gest tak­ing a look at a list of best prac­tices for long-term stor­age on blu-ray discs as com­piled by Brian To­masik. Like him, I am not an ex­pert ei­ther, so take this ad­vice with a grain of salt.