Recent Database Migration—Report Bugs
Hey all,
LessWrong is currently in the process of migrating databases (from mongo to postgres). Today we migrated a large number of important collections, including Posts, Comments, Tags, Votes, and others related to those. If you notice any new bugs related to posts, comments, tags, or voting, please let us know ASAP!
The best way to ping us is via Intercom—unless you’ve hidden it, there should be an icon on the bottom-right of your screen (on desktop views):
You can also post here.
(Please also report existing bugs, if you’re aware of any, just let us know if you’re doing so—new bugs are more likely to be related to the migration and are more likely to be time-sensitive.)
Known Issues
Read Statuses
Migration for these only partially succeeded. Recovery in progress (last attempt briefly took the site down).
Comments Counts on Post Items
Outdated. Will try to get this fixed after Read Statuses.
Just curious: what motivated the transition?
Some combination of:
tech debt from design decisions which made sense when rebooting as LW 2.0, but have become increasingly unwieldy as development’s continued
strictly speaking there were options here that weren’t moving off of mongo, but it’d be much more difficult to make sane design choices with mongo involved. the fact that it can do a thing doesn’t mean it does that thing well.
mongo’s indexing in particular is quite bad, both in terms of how finnicky mongo is about whether it can figure out how to use an index for a given query… and then also limiting you to 64 indexes per collection. (why would you need even 64 indexes per collection, you ask? let’s just say that development around here has been focused more on shipping things quickly than on other things… and, to be fair, we only got there with one collection. of course, not all of those indexes are used very much, but another mongo headache is the insane, poorly-documented locking behavior when you try to drop an index, which causes outages. lol.)
the EA forum team (particularly @ollieetherington) doing much of the heavy lifting in terms of writing the infrastructure necessary to migrate
in a vacuum I’d unquestionably rather be on postgres, but it’s not clear that it would’ve been worth the effort if it’d just been the LessWrong team and our use-case(s) at stake.