Personal information management
Several weeks ago, I began using personal wiki software Zim Wiki (free and cross-platform for Linux & Windows; I recommend nvALT on Mac OS X) to record all of my notes-to-self. I’ve found it to be a nice software tool for implementing some of the effectiveness advice I’ve read on Less Wrong. This post is a fairly personal overview of my usage.
I looked at a lot of personal information managers before choosing Zim. Here are the features that caused me to choose it over the other software I looked at:
-
Probably the most important feature: Jump-to-note capability with autocomplete. Pressing Control-J gives a text box. Start typing in the text box and it autocompletes with the names of any of the notes in my notebook (or allows me to create a new note). This is the proverbial 10% of the feature set that provides 90% of the benefit over scattered text files. Opening a specific note to add another thought or idea to it is a very common operation for me and this feature makes it very quick. Only a few tools I’ve found seem to have comparable functionality: WikidPad (with Control-O), and the Notational Velocity family of information managers kind of have it. (For Notational Velocity/nvALT, I recommend coming up with some kind of namespacing scheme so note names collide with note text less frequently in your searches. For example, I prepend reminders for future situations with “f.”, journal notes with “j.”, policy notes with “p.”, Less Wrong post drafts with “l.”, etc. Then command-L works as a pretty good “jump to note” shortcut.)
-
Pressing Control-D, then pressing return inserts a timestamp at the position of my cursor. This has been useful for a variety of logging-type applications. (I replicated the same thing with nvALT on OS X with aText.)
-
Zim is a desktop application. This has a couple advantages:
-
I configured a keyboard shortcut to open it, or bring it to the front if it was already open, using a modified version of the Linux shell script in this forum thread. (Alfred is nice for this on OS X.)
-
All my notes are stored as plain text files on my hard drive. I keep them under version control, which lets me do things like answer the question “what new ideas for becoming more effective have I had over the past week?” (I didn’t use the built-in version control plugin because I found its UI glitchy.)
-
-
There’s inter-note linking capability, also with an autocompletion dialogue. (Press Control-L to create a link.)
-
Moving through note browsing history can be done with Alt-Left and Alt-Right.
It starts fast.
Notes are saved automatically, hierarchical note organization is possible, backlinks are tracked, incremental keyword search within a note is possible, and there appear to be a variety of other features I haven’t yet had a chance to abuse.
Using Zim has meant a really low level of friction for writing new stuff and retrieving/reading/adding to stuff I wrote. I’ve been using it about a month and I’ve got ~46K words in it, which seems to be around the length of a short novel. RescueTime says I use it 4-8 hours per week. Some stuff I’m using it for:
-
Strategizing. There’s something kind of calming about writing my thoughts out when I’m choosing between several options or trying to figure out what to do. I suspect that as soon as the amount of information related to a decision exceeds the capacity of my working memory, I benefit from writing stuff down.
-
Recording business ideas, self-experimentation ideas, essay ideas, etc.
-
Making plans and filing away notes related to future circumstances.
-
Taking notes related to software I’m developing.
It’s hard to measure how much benefit I’m getting out of all this, though it feels pretty useful. I’m inclined to agree with Paul Graham:
...actually there is something druglike about [the notebook and pen], in the sense that their main purpose is to make me feel better. I hardly ever go back and read stuff I write down in notebooks. It’s just that if I can’t write things down, worrying about remembering one idea gets in the way of having the next. Pen and paper wick ideas.
- What makes people intellectually active? by (29 Dec 2018 22:29 UTC; 115 points)
- Thoughts on designing policies for oneself by (28 Nov 2012 1:27 UTC; 112 points)
- 's comment on Identification of Force Multipliers for Success by (21 Jun 2014 5:57 UTC; 29 points)
- 's comment on CFAR’s new focus, and AI Safety by (3 Dec 2016 12:23 UTC; 28 points)
- Managing one’s memory effectively by (6 Jun 2014 17:39 UTC; 26 points)
- What is your Personal Knowledge Management system? by (16 Jul 2019 20:20 UTC; 22 points)
- Productivity thoughts from Matt Fallshaw by (21 Aug 2014 5:05 UTC; 19 points)
- 's comment on Learning Optimization by (29 Apr 2015 13:38 UTC; 14 points)
- 's comment on Paper Trauma by (10 Feb 2018 23:09 UTC; 12 points)
- Empirical claims, preference claims, and attitude claims by (15 Nov 2012 19:41 UTC; 12 points)
- 's comment on The Growth of My Pessimism: Transhumanism, Immortalism, Effective Altruism. by (30 Nov 2015 9:34 UTC; 8 points)
- 's comment on Regular Lesswronger’s interviews by (14 Jun 2015 5:17 UTC; 6 points)
- 's comment on Seeding a productive culture: a working hypothesis by (20 Oct 2017 5:45 UTC; 5 points)
- 's comment on Effectively Less Altruistically Wrong Codex by (18 Jun 2015 0:54 UTC; 3 points)
- 's comment on Tiddlywiki for organizing notes and research by (2 Sep 2019 22:20 UTC; 3 points)
- 's comment on Open Thread: how do you look for information? by (8 May 2013 20:50 UTC; 1 point)
Anyone using org-mode here? It’s free, cross-platform, and also has links (to arbitary files!), outlines (actually, the whole thing is mostly about hierarchical headings), you can use it using mostly the keyboard only, and there are also some Android / iPhone apps (however, I haven’t tried them yet).
It has the added benefit that headings have nice colors (especially with a white on black color theme), so if you put it on full screen everyone gets the impression that you’re doing something complicated and useful thing. (Even if you’re just churning out notes about how to improve your time management as a quite nice & recursive way of procrastination.)
(And yes, it’s an emacs mode, actually this is the reason I ended up using emacs for all kinds of other things...)
Additional note: linking to all kinds of files can be an awesome tool when building maps of big and ugly software systems.
I have probably sunk something like 300 hours in org-mode and ultimately abandoned it in favor of a system incorporating Evernote and Nozbe. Org-mode has been a source of much frustration for me. It seems so great, it seems to have all the features one could ever want, but every time I’ve tried to implement it (three separate attempts, each time starting from scratch and thinking I knew “what I was doing wrong last time,”) the system has grown huge and unweildly, leaky and unreliable, and missing key features that I needed.
On the plus side I learned how to use emacs really well.
edited to add: The iPhone app is pretty bad, for the following reasons: It is ugly and navigation is unintuitive, and the text-wrapping is essentially broken. Furthermore, you have to manually synchronize every little thing you do both pushing and pulling to your central repository or you’ll quickly end up with inconsistencies which are a disproportionately huge pain to correct.
I actually still use org-mode if I’m simply going to be outlining a complicated project, but I’ve given up on using it as a task manager. I really wanted to like org-mode.
Late the party, and actually found this thread googling around for “Org-mode file/organization strategies.” I’ve been using Org exclusively for work notes, and am finding myself in a similar situation re. being unwieldy. I constantly struggle with choosing one file per project, one big file with one headline per project, or files dedicated by type (one for todos, one for daily journal logs of experiments/efforts, references, etc.).
Org seems like it should be great for moving stuff around, but I find it not that easy. Refiling a mess of headlines seems to be cumbersome, and how do I know that my new strategy will last/work?
I’d love to know how Evernote solves the unwieldy issue for you. I’ve tried Evernote, Wunderlist, TiddlyWiki, todo.sh, Zim, and I’m sure others I’m not remembering.
What I’ll never give up is the ability to intersperse prose and code. I love, love, love writing all my work reports with embedded R code for analyses in Org-mode, exporting to really nice looking PDF reports. Super awesome, and soooo easy vs. writing all the code elsewhere to generate plots and then inserting them one by one into a ppt. In that respect, Org is awesome. I just haven’t figured out an information hierarchy/taxonomy that makes me happy.
It looks like I wrote the grandparent comment over two years ago and I am still primarily using Evernote and Nozbe. Evernote is invaluable for its ability to capture practically any form of information very quickly and then search it later. I can also intersperse “capture” items like reminders with “work” items like drafts of writing.
Nozbe is a fully functional GTD application and it’s the backbone of how I manage my tasks.
Theoretically org-mode is great because it combines capture with workspace, but in practice I always found it impossible to smoothly transfer between those two functions.
I tried the Android app just after I read your comment (it’s a thing I’ve been putting off for a long time), well… it really doesn’t include the “creating nested outlines easily” part I like org-mode for, and the synchronization part also seems to be kind of… strange. Just as you said.
What I really like about it is the minimum effort that it needs to, for example, create a todo item (compared with web-based solutions). Too bad that these todo items usually end up really unorganized. Would be indeed nice to have some interface between, e.g. Nozbe and org-mode, and use each of them for the task it is better suited for.
(I also agree with your point about learning emacs really well… or in my case, at a relatively acceptable level :))
As much as I like org-mode (and I like it so much that I don’t see myself changing systems unless someone comes along and refines the hell out of org-mode under a new name), I’ve wished for more from it. Perhaps I ought to just get to work learning more emacs, but some trivial inconveniences and vague desires I’ve encountered so far:
I’ve not been able to get the Android app working. Ever. Maybe if I go back and put 40 more minutes into getting it to work.
I’d love to work with a high-contrast, white-on-black background. I’ve still not learned how to do this, even with a fair bit of looking
If I could switch to a full-screen (not just maximized, but distraction-free, maximum visibility full-screen) that would be fantastic.
I sync my org-file (yeah, I only use one, and it’s huge) with Dropbox. Don’t have to worry about losing it if anything goes dead. But I really just wish that Evernote did all the same stuff. Because Evernote has a slick look, and a pretty great Android app. And I’d be willing to pay money for such a slick implementation of my huge org-file.
I use Astrid as a todo-manager. It’s not a bad system (even though I’ve had some obnoxious database corruption issues every month or so), but what I’d really like is a similar system (and there are so many todo managers out there that have the same features, and they’re always adding more) that could just read the TODOs out of my org-file and append the appropriate tags, etc. I could see myself someday knowing enough emacs lisp to figure this out on the org-mode end, but right now I don’t plan on learning enough programming to work this out as a supplementary app, and it looks far, far simpler to just offer a bounty on this kind of thing that someone else can program.
Oh, yes. I do Dropbox syncing, too (this is the other good thing about org-mode: plain text files). And there might be some truth in the statement that while org-mode is excellent for a single file, things start to be less seamless when it comes to more of them… inter-file links don’t seem to be that reliable, for example. Is this the reason for your One Big Org File?
For white on black, it’s just (setq default-frame-alist ’((background-color . “black”) (foreground-color . “white”))) in your .emacs.
Actually, it’s kind of typical lesswrong that I started off with a comment popularizing org-mode, but ended up changing my mind about it (well… kind of), the newest experiments include Notational Velocity (they seem to be good at the global search stuff org-mode is lacking, but not so nice indented lists locally), and also this system:
http://www.speakeasy.org/~lion/nb/book.pdf
which includes paper notebooks, maps of your thoughts and similar fancy stuff, but I haven’t yet finished reading it (it’s long and not exactly the most organized stuff I’ve ever read… but it has good ideas.)
For links, I switched to the org-id module and a unique ID for any new links. It works as long as the file containing the target headline is in the same directory as the file containing the link.
Yes, and mostly love it. Just not happy with the structure of my information management strategies, at least for daily work documentation. Put “X” under the specific project I’m doing it for? Or what if the learning seems more general… should i start a new tree for longer-term reference knowledge? Or summarize the specific knowledge more generally and keep a copy of both in separate areas? Or write only one and link to it in the other?
Stuff like that.
Other benefits I’ve really appreciated:
embedded/executable code blocks. This is my favorite, favorite feature, and I will never go back (if I can help it) to running analysis code (I do a lot of data analysis/viz with R/ggplot2), generating plots, and then having to insert them into a .doc or .ppt.
on that note, being able to export in general is a fantastic feature of Org. HTML, PDF, markdown, whatever. I love not having to futz with image placement manually. Get something you like with some #+attr_html/latex arguments, and then insert a bunch of images really easily by just linking to their location on disk.
org is the only, or one of the only, applications that lets you mix and match notes and todos. I used to use TiddlyWiki for all of my work notes and really liked it. But I had to manage todos in something else. But why? Todos often come about in the context of notes, say in a meeting, or for some home project. I really like being able to keep the todos in Org right with whatever prompted the need for action (then view just the actions you need to complete with agenda views).
Clocking/time tracking is another aspect that seems awesome, but I haven’t started using. I’d love to get to the point where I record what I work on, and especially an estimate + logged hours. Not sure what I’d do with the data, but having it vs. not at least makes post-analysis a possibility.
Still, my various attempts at org file structure seem to end up cluttered and with things structured really oddly. I’ll kick off a project with my estimate of what “categories or knowledge” it will require, and as the months or years go on, I’ll be in a rush and just resort to keeping a date tree and stuffing the stuff in there like a journal instead. Now I have project-specific info scattered around through monthly journal trees. Harder to archive/find, and end up unfolding a bunch of headlines to find stuff.
Anyway, neat to find other users, so I thought I’d comment even though I’m really late to do so!
I don’t use zim wiki, but I do have page just like that: notes and thoughts
Currently, it contains ideas like: fear incubation, legoization, serendipitous incentive as well essay like “A Healthcare Ancedote in America”, “Self-Quantification”, and “Why Choose Prosthesis”. They have lot of grammar mistakes and other type of writing bugs as well. I need to get better at learning grammar.
A page like that helps me remember lot of ideas and solidify them into essays, hacks, and other products. So far, it only help me learn information and write essay, but that’s because I don’t have any solid hardware skill yet. Before, I thought I couldn’t write essays, but that’s because my ideas and citations are not gathered and remembered.
I use Catch (catch.com). It’s through the web, and on android. It has an open API, https://catch.com/developer/ so if you wanted to wrap your own SRS system around it, you could.
Catch.com is shutting down 2013-08-31. You can download a zip file of your notes.
That sounds pretty great. I happened to get a free copy of OneNote 2010 a little over two years ago, and it’s played a similar role for me. It has everything you mention above, including the autocomplete search/jump which I agree is killer. I think linking and timestamp don’t come with keyboard shortcuts, but you can add shortcuts for any command [edit: see pjeby’s reply—thanks!]. A couple other features I appreciate: constant autosave, and a keyboard shortcut for taking screen clippings when OneNote’s running in the background. There’s even good math support, although I still use LaTeX for math/physics notes.
The downsides: It’s not free, open-source, or cross-platform. There are Android and iOS apps, although I’d need Android 2.3+, so I can’t yet comment on that. My least favorite aspect is that it doesn’t store things in plaintext, although you can get plaintext out with a little bit of work. (Probably easiest to save a notebook as html and go from there.) Maybe part of the reason is that there’s lots you can do that isn’t text (more flexible formatting than e.g. Word, recording and inserting video/audio notes, drawing, tags, integration with Outlook tasks) but much of that could at least be formatted sanely, and I don’t use it anyway. Zim Wiki sounds like a good choice if I ever get frustrated with the lock-in.
Keyboard shortcuts for OneNote:
Alt-Shift-T inserts time
Alt-Shift-D inserts date
Alt-Shift-F inserts date+time
Ctrl-K opens a link-creation dialog (or converts existing text into a link)
Surrounding text in
[[double brackets]]makes a wiki-style link, creating a new page with the enclosed title (or linking to an existing one) in the same section.Also, any page or paragraph can be a link target, with its own onenote:// URL that can be launched from other programs that allow links to be embedded.
Yeah, if you happen to be on Windows and have OneNote 2010, it can be pretty sweet for this sort of thing.
I used to run a wiki (instiki) on my local machine to do something like this, but I eventually gave up reinstalling it on new machines after running into problems with data compatibility (unlike Zip, it isn’t stored in plain text files).
Now I just use a plain text file for my todo list, and either google docs or drafts in gmail for my various personal notes; I’m not extremely satisfied with it either—Google docs is convenient for access from anywhere, and it’s nice to be able to use spreadsheets, but I miss the ability to easily create many small interlinked pages.
Do you use Zim on several machines, synching the text files from one to another with source control?
No, I just use it on one machine.
Just started using it, haven’t done anything beyond the basic note writing so far. Is there a better source of instructions than the main page? Its somewhat confusing.
I mainly figured stuff out by poking around in the menus, I think. If you have specific questions, I ought to be able to answer them.
Are you still using Zim Wiki?
I switched my OS to Mac OS X and started using http://brettterpstra.com/projects/nvalt/ However, there are features I miss from Zim Wiki. I would definitely recommend it on Linux.
(BTW if anyone is ever making the same Zim → nvALT migration get in touch with me… I wrote some scripts to make it easier.)
What percentage of the written text do you re-read, or update regularly?
I’m guessing that roughly 15% of my notebook is relatively “high-traffic” stuff that I actually review/rewrite on a semiregular basis (once every few weeks at least).
“Brain support”? Sensationalist title.
improving the efficiency of the exobrain tools we use seems to be vastly underrated.
I’m not so sure of that—there’s also a risk of spending too much time and attention on tools rather than on actually getting stuff done—what’s called “Productivity Porn”.
A bit like if an online community spent all it’s energy on talking about how to have quality discussions, instead of actually having quality discussions.
I agree but it seems like a niche problem. How many people have a good note taking system and understand how to use search engines to full effectiveness?
Zim doesn’t represent any improvement in the efficiency of taking computer notes. Evernote beats it on most features.
Is “thought support” OK?
You still don’t give any arguments to support such a claim (to wit, “Zim Wiki provides thought support”), so no.
I suggest “Zim Wiki made me feel more productive” or “Zim Wiki is pretty cool, give it a try”
I default to interpreting it as “keeping a brain alive with machinery.”
I might use something like this, except I find it extremely useful to be able to note things down on my smartphone wherever I am when I have a sudden thought, and have that synced with my other devices.
So I just use Evernote. Wish it had source-control-ish features, though.