tech debt from design decisions which made sense when rebooting as LW 2.0, but have become increasingly unwieldy as development’s continued
strictly speaking there were options here that weren’t moving off of mongo, but it’d be much more difficult to make sane design choices with mongo involved. the fact that it can do a thing doesn’t mean it does that thing well.
mongo’s indexing in particular is quite bad, both in terms of how finnicky mongo is about whether it can figure out how to use an index for a given query… and then also limiting you to 64 indexes per collection. (why would you need even 64 indexes per collection, you ask? let’s just say that development around here has been focused more on shipping things quickly than on other things… and, to be fair, we only got there with one collection. of course, not all of those indexes are used very much, but another mongo headache is the insane, poorly-documented locking behavior when you try to drop an index, which causes outages. lol.)
the EA forum team (particularly @ollieetherington) doing much of the heavy lifting in terms of writing the infrastructure necessary to migrate
in a vacuum I’d unquestionably rather be on postgres, but it’s not clear that it would’ve been worth the effort if it’d just been the LessWrong team and our use-case(s) at stake.
Just curious: what motivated the transition?
Some combination of:
tech debt from design decisions which made sense when rebooting as LW 2.0, but have become increasingly unwieldy as development’s continued
strictly speaking there were options here that weren’t moving off of mongo, but it’d be much more difficult to make sane design choices with mongo involved. the fact that it can do a thing doesn’t mean it does that thing well.
mongo’s indexing in particular is quite bad, both in terms of how finnicky mongo is about whether it can figure out how to use an index for a given query… and then also limiting you to 64 indexes per collection. (why would you need even 64 indexes per collection, you ask? let’s just say that development around here has been focused more on shipping things quickly than on other things… and, to be fair, we only got there with one collection. of course, not all of those indexes are used very much, but another mongo headache is the insane, poorly-documented locking behavior when you try to drop an index, which causes outages. lol.)
the EA forum team (particularly @ollieetherington) doing much of the heavy lifting in terms of writing the infrastructure necessary to migrate
in a vacuum I’d unquestionably rather be on postgres, but it’s not clear that it would’ve been worth the effort if it’d just been the LessWrong team and our use-case(s) at stake.