Personal knowledge management (PKM) is a collection of processes that a person uses to gather, classify, store, search, retrieve and share knowledge in their daily activities (Grundspenkis 2007) and the way in which these processes support work activities (Wright 2005). It is a response to the idea that knowledge workers need to be responsible for their own growth and learning (Smedley 2009). It is a bottom-up approach to knowledge management (KM) (Pollard 2008).
I am curious about how LessWrongers manage personal information, including but not limited to—research reading/ research output, side-projects/ hobbies, blogging, managing watchlist/ to-read list, social information, incremental-reading, making flashcards, threads-from-unfinished-conversations, braindumps, and maybe even general GTD-stuff.
Information about any of the following will be greatly appreciated.
-
General framework/ philosophy for PKM. Can be very concrete and well-defined like Building a Second Brain or a very a general rule-of-thumb like ‘I bookmark everything/ I write down all thoughts in org-mode.’
-
Toolchain. What and How. Everything counts—wikis, org-mode, physical notebook, etc.
-
Workflow. Both local (how you manage fuzzy and perpetually arriving information on a day-to-day basis) and global (how you PKM for bigger and concrete projects over longer time).
-
Comparison of multiple frameworks, if used.
-
Tips and tricks/ info about building PKM habits.
Principles:
Capture everything: Do not assume your brain will remember anything. Write it down ASAP. Use whatever will let you capture it quickest, whether that’s pen and paper or a digital solution.
Review and process: Make sure you actually look at the things you wrote down regularly and organize it. If it was an idea you need to act on (e.g. a topic for a blog post, or a reminder to look up a particular concept), add it to your task manager. If it was a thought for reference, add it to your ‘second brain’/note-taking/archive system, and add tags so you can find it easily later. Once processed, archive or delete the item from your capture system.
“Everything should be made as simple as possible, but no simpler”: There is a balance to strike between using as few tools as possible, and using many tools that are each specialized to do one thing really well (a tool that tries to do everything tends to do nothing well).
Consider sustainability: In an ideal world, I would only use open-source, non-proprietary tools. But often commercial tools do a better job and/or have a nicer interface. In these cases, ensure that you can export your data at any time in a standard format (e.g. markdown, xml). Also, be prepared to pay a subscription fee—this helps keep the tool going!
What I use:
For capture: Google Keep. If it’s a task, sometimes I add it straight to my task manager (Asana).
Info sources: I use Feedly to subscribe to lots of RSS feeds. When an article comes in that I want to read, I add it to my ‘read later’ board. I also use the ‘Save to Feedly board’ Chrome plugin to save any article I find on the web to ‘read later’. The goal is to maintain inbox zero in my ‘read later’ board—I remove an item once it’s read, so I know that anything in there is unread/unprocessed. I use Goodreads to manage my books, and Castbox for podcasts. I get free e-books and audiobooks from the library, and I also have an Audible account.
For archive, reference, notes, and high-level planning: Notion. I use it as a personal wiki. It is very flexible, easy-to-use, and has relational databases(!!) If the mobile app was faster it would replace Google Keep as a capture method for me. It could also in theory become a task manager (using the database feature) but it currently lacks some key features (like recurring tasks). FYI, you’ll likely need to subscribe to the pro version to get enough space. I think it’s worth it.
Do you mind sharing your Notion setup? I’ve been thinking about setting up a personal wiki with it, but I’m unsure of how to best structure it overall. I’ve just started a bare-bones system for planning and task management, but I’m still getting the hang of using Notion effectively.
I’m constantly experimenting with it! The downside of it being so flexible is that it can take a while to figure out the best system.
At the moment, everything goes into one database called ‘Notes’. I enabled the ‘Created at’ and ‘Edited at’ properties. I also have multi-select properties for themes (e.g. rationality, productivity, economics, etc) and for type (e.g. random thought, blog idea, resource, article, tool, etc). I also have a checkbox property called ‘processed’ - and I filter the view of the table to only see the unticked items. Everything I add is by default ‘unprocessed’ (i.e. unticked) - this allows me to add stuff from the web (with the web clipper) and quick random thoughts without worrying about immediately sorting them. Every so often, I go through everything that is ‘unprocessed’ and sort it (add tags, finish reading it, add highlights or more notes, links to other notes, etc) and once I’m done I’ll tick the ‘processed’ box so it’s hidden from the default view.
I ‘favorite’ the notes I use most regularly (e.g. I have one called ‘useful info’ which has stuff like my health insurance number, wifi passwords, etc). Otherwise, I navigate by searching, or by filtering on certain tags.
I’m planning to gradually build this out into a relational system (e.g. creating a ‘project’ table and linking the notes to relevant projects, etc). I try to ‘organize opportunistically’ (as described in Part II C here—retrieved this from my Notes table in Notion!) as I find most of my attempts to impose a top-down structure are not flexible enough.
For recurring tasks, I use checkvist.
You can embed your lists into Notion pages.
Vim + markdown (evolved into standardized DSL for note-taking) + git.
What I’m here to say is that programmers’ tooling is a great fit for knowledge management. Fuzzy file finding / fuzzy grep is something you can get for free with plugins, and any interaction like going-to-previous-file-by-date can be easily achieved with editor scripting. Version control (Git in this case) allows per-line editing history & multi-machine granular merge and basically adds another dimension to text editing: time. All versions of the text exist in their own right and can be inspected and rolled back to. I use heavily optimized editor keys for 2-3 keypress commit-related operations, most of it came for free with plugins. The power of Vim for text editing is not to be underestimated as well.
Another thing perhaps worth mentioning is tiling window management + tmux. I have tmux / terminal sessions for different knowledge-management tasks (notetaking while reading an article is a different tmux session than daily agenda e.g.) bound to key combinations so that they can be accessed with low latency. The lowest-latency inbox is just a text field that appends to a file, for use when any distraction is to be avoided.
Basically my point here is, learning Unix is high value, because no premade tooling can be as well-designed as a hand rolled one, for under-explored domains.
Entirely seconding the bit about hand-rolled tooling, with the caveat that there is a lot of hand-rolling you can do without learning Unix. (This is not to denigrate the value of Unix, but only to avert the possibility of someone reading this and thinking “oh, but learning Unix sounds hard and is not really for me, I guess hand-rolling my tools isn’t an option”—it very much is! There are many paths to DIY.)
Agree. Reason I used the term Unix is because of it’s famous philosophy of composable tools and plain text.
E.g. the frequency + recency filter that I use is a separate program, that can be in turn applied in different parts of the system (e.g. before feeding the input to fuzzy select tool like fzf, but not necessarily): https://github.com/ccheek21/fre
I love how you emphasized learning Unix tools. I use other things mentioned here except tmux. Would you be willing to share your tmux workflow in more detail with keybindings?
Here’s .tmux.conf, however it mostly covers the in-tmux things like split/tab management (e.g. I open & switch to new tabs with alt-1/2/… instead of default C-b 1/2/… This mirrors the browser behavior and is 1 less keypress):
https://github.com/mwgkgk/dotfiles/blob/master/tmux/.tmux.conf
Tmux allows neat tricks like sending a window between sessions or sending keypresses to a session. E.g. I have a script called “portal” that opens a new window in a target tmux session (that we’re opening a “portal” to) with the current directory, and brings that window to the foreground.
Another benefit of tmux is that all of my editor sessions are independent of Xorg and so can survive a restart of X or be reused from a different X session (e.g. when testing a WM).
Here’s sort of a teaser of which tmux / urxvt sessions I have bound in sxhkd (some are still bound from dwm config):
https://github.com/mwgkgk/dotfiles/blob/master/sxhkd/sxhkdrc
The launchers themselves (e.g. “ship”, “tower”, “girl”) are unfortunately not online at this point. What these files do is open (with few exceptions) a floating window with the named tmux session and bring it to front, or run the args in a new tmux window of the target session. These are the different-purpose knowledge-management sessions I was referring to.
Between those are 2 firefox sessions, which is another thing perhaps worth mentioning. I run 8 thunderbird sessions with RSS feeds and and ~20 firefox sessions. Two of those found in sxhkd config are floating, for quick anonymous / non-anonymous-programming-related lookups. I find separating the browser sessions very valuable for honing the suggestion streams that google / youtube / … throw at us.
Emacs + org-mode + Dropbox
1. When learning new things, I use the principles of deep learning to link what I’ve learned to other things I’ve learned. This helps me remember the important bits and usefully be able to apply them when they’re relevant.
2. When working on a project, I take all the things I’ve learned that are useful and put them into TheBrain, linking them to relevant parts of the project as well as previous types of knowledge they’re related to. The types of knowledge I try to capture in the brain are:
Mental Models (Ways of Thinking That Are Useful)
Processes (Steps/Flowcharts That Are Useful)
Modelling Tools (Specific Ways of Modelling Knowledge and Information That Are Useful)
Power Tools (Halfway Between a Mental Model and a Process, a Trick or Technique or Idea That is Particularly Useful)
Questions (Things I can ask myself)
By consistently doing this with new projects, over time I develop an interlocked set of knowledge that makes it very easy for me to find relevant tools for my new projects.
3. When I find a particularly useful article/image/etc, I add it into Evernote. I can then search Evernote when I remember a particularly useful article but didn’t fully internalize the knowledge or remember the specifics. This can be added to the brain if needed, or sent to other people when they’re dealing with relevant problems.
4. I take notes on books in my Kindle, and sync them to Evernote with clippings.io
5. I used to take Shallow knowledge that was particularly important and add it to Anki, but it was very hard to find shallow knowledge that I knew in advance would be particularly important, and gave up on the practice in the favor of just saving important thing in Evernote. I try to err on the side of savings things, and am always frustrated when I’m trying to remember an article but didn’t save it in Evernote..
Tools:
I use Pinboard to manage my bookmarks.
I have a personal wiki (an instance of my custom PmWiki-based wiki platform) that I can access from anywhere, to store ideas, to-do lists, essays in progress, brainstorming, etc., etc.
I use a PDA (a Palm IIIxe, synced to one of my computers) to jot things down.
Tips/tricks:
Find reliable, robust, flexible tools that will last a long time, and are adaptable to your needs.
When it comes to software, avoid the proprietary and the centralized. Where it is unavoidable, look for what is simplest, has a proven track record, and which can export your data in standard formats (and do this regularly).
Don’t overthink it. Don’t start by investing a lot of effort into some fancy workflow or some fancy philosophy. Avoid the urge to yak-shave. Write it down first, organize it later.
I endorse the tips/tricks section here (and it seems like the most important bit because different individuals have idiosyncrasies that make different tools useful).
Yes, the “tools” section of my answer is simply an example, if you like, of tools which (a) fit the criteria I note in the “tips/tricks” section, and (b) fit my specific needs. Others’ needs may be different, so the tools they pick may be different also, but my advice is general.
Extended example, in case anyone is interested:
Before I found PmWiki and built my wiki platform, I wrote everything down in RTF files (edited with nothing more fancy than TextEdit), synced between my various computers via Dropbox.
Note that “RTF files, edited with TextEdit” fits the criteria: RTF is an extremely common format, TextEdit is decades old but constantly maintained (and anyway you can edit RTFs with basically anything), the data was in my possession, convertible to anything, easy to back up, easy to sync, easy to search.
(I have since migrated most of those files to the wiki, which also fits the criteria, in different ways. It was easy to migrate. “Text, saved in some standard format” is something you can always find some way to usefully work with.)
The important thing was that the way I started doing this was just:
Open TextEdit.
Begin typing.
The biggest hurdle is starting. The second biggest hurdle is continuing. Everything else is secondary. Worry about tools, workflows, philosophies, etc., after you’ve got enough content for any of those other things to matter.
I use Notion.so. I mostly use it like a wiki, but I find the rich formatting and easy move-ability of the blocks to be helpful. I also use the database features to collect notes for ongoing projects, using it more like a journal. Notion is slow on mobile, but I find that taking the time to transfer bookmarks and insights to Notion helps consolidate them.
For organizing ideas that have a temporal component, I use preceden.com timeline. This is great for keeping track of books I’ve read and for medium and long term planning.
For longer thoughts and writing I use Google docs. I use Google Drawings for mindmaps and conceptual diagrams. I then link to the docs and drawings from Notion.
For PDF articles I use Notability on my iPad. This has excellent highlighting and note-taking features.
For ebook reading and organization on the iPad, I use Kybook.
For video lectures Youtube playlists, with youtube-dl gui for offline viewing.
I have been wishing for a long time for a fully integrated solution, but each tool has it’s strengths and weaknesses.
I’ll repeat the endorsements of org-mode, and add some links to specific org-mode features that I use.
I use org-capture to capture to-do items. I’ve found nothing quite as streamlined as being able to hit
C-c c tand just type what I need to doI use org-agenda to create daily to-do lists for myself. When I org-capture a new to-do item, I will add a deadline (
C-c C-d) to indicate when I want to do the task.I use org-publish to write my website. This is nice because it’s very immediate for me to write, and I can hit
C-c C-e P xto publish a new version of my websiteFor other org-mode tips, you can look at my emacs tips page, which has my emacs configuration documented (in a somewhat haphazard and idiosyncratic fashion)
I’ve tried other PIM tools, like vimwiki, workflowy, DynaList and Dropbox Paper, and I’ve found that of all of them org mode offers the right mix of customizability and immediacy for me. That said, I’ll be the first to admit that org-mode doesn’t have the easiest learning curve, and it’s support for mobile devices is pretty much trash. (People have recommended orgzly, but, honestly, orgzly’s UI pretty terrible.) What I do when I’m out and about is capture notes in Google Keep and then copy those notes over into org-mode when I get back to my computer.
VS Code + Extensions (Markdown + Latex parsing + Graphs parsing + custom snippets).
Anki
Chromium Extensions (Super Simple Highlighter)
Git
Calibre (Books manager also I usually convert books to HTML)
This “stack” is very useful for me.
I read books on multiple devices—GNU/Linux, Android, and Kindle. Last time I checked, Calibre was too feature-rich and heavy, but lacked a simple getting-out-of-my way workflow for syncing my reading between devices. Is there a better solution now?
Calibre is great for me when syncing epub/mobi books from my computer to my Kindle Paperwhite (I don’t think I’ve ever encountered a major problem in this particular process, only on very old books which it has trouble converting). Besides that I use it to convert epub/mobi books onto html which is where I like to read when in my computer (using the browser, chromiun or firefox, which means I don’t use the screen reader of Calibre, this way I can make easy modifications with css and inject css to highlight the most important parts with several different encodings allowed by the extension I talked about in my answer).
It’s too feature-rich and heavy and it gets in your way, but it solves many simple problem if you use only some of its features.
This is the open-source repository and I recommend always being up-to-date with the latest release (that may solve some of your problems). Besides that I really recommend Calibre, it’s an essential tool for my purposes.
Workflowy. Dynalist and others have more features, but I don’t want more features. More features means more decisions. I organize by month and do a review at the end of months, plus a year end review when collapsing into my archive tab. Tags for things like book notes, quotes, routines etc.
I wrote about mine here. Still using it. Contra Romeo, I love features. Sometimes I add my own by writing Python scripts that interact with my notebook.
No particular philosophy: just add some kludge to make your life easier, then repeat until they blot out the Sun.
Non-computer tool is paper for notes & pen, filing everything useful to inbox during daily review. Everything else is based off org-mode, with Orgzly on mobile. Syncing over SFTP, not a cloud person.
Wrote an RSS reader in Python for filling inbox, along with org-capture. Wouldn’t recommend the same approach, since elfeed should do the same reasonably easy. Having a script helps since running it automatically nightly + before daily review fills up inbox enough novel stuff to motivate going through it, and avoid binging on other sites.
Other than inbox have a project list & calendar within emacs. Not maintaining a good discipline for weekly/monthly reviews, but much smoother than keeping it in your head.
I have a log file that org-mode keeps in order by date. And references file that don’t get very organized or used often. Soon will try to link contents of my massive folder of PDFs with it.