Systemizing and Hacking

Let’s call two broad clusters of problem-solving “systemizing” and “hacking.” Systemizing roughly refers to building general, robust solutions. Hacking refers to quickly building one-offs. You would expect to accrue technical debt while hacking and pay it down while systemizing.

Both are useful in different situations. A big win for systemizing would be finding patterns in code and dramatically reducing the complexity by factoring out those patterns. A big win for hacking would be quickly testing several UIs to find one that potential customers love.

As you can probably guess, I lean very heavily towards systemizing, and have typically sucked at hacking – I generally relied on coworkers for that skill. But I started working on solo projects where I couldn’t easily rely on someone else, so naturally, I developed a system for hacking things together.

The concrete example here is that I wanted to be able to run thousands of completely independent machine learning jobs quickly in parallel across a lot of computers, and I was looking into AWS Batch to do this.

Here’s how I would have approached this in “systemizing” mode:

  1. Break down all the parts involved, e.g. AWS Batch, Docker, boto3, etc.

  2. Study them in detail until I grok each one, writing small bits of code primarily designed to help me understand

  3. Assemble a solution

Here’s how I did this in hacking mode:

  1. Find a valuable point of high uncertainty (e.g., running Python scripts through Docker)

  2. Find something that does something similar to what I want (e.g., an online introduction to Docker that involved running a particular Python script)

  3. Modify that until it does what I want (successfully running my Python script and uploading data to AWS)

  4. Find the next valuable point of uncertainty and repeat (running a single Docker image on AWS Batch)

This was really effective – previously, I might have spent ~40 hours deeply learning all the tools involved before I got the system up and running. Using the hacking approach, it only took me about 3 hours – literally an order of magnitude faster.

Of course, the hacking approach has clear disadvantages – I don’t deeply understand Docker or any of the tools involved, and my experience here won’t save me a ton of time if I work on a similar project. But in this case, I don’t expect to do anything similar in the near-term, so hacking was a huge win. More generally, having the skill of hacking things together gives me many more options for projects in the future.