I did have a sense that it wasn’t The Rails Way, but I also had a sense that it was quite readable and sufficient. The code was working well locally, so I issued a pull request.
When I issued the PR, someone pointed out a major issue.SmsLog.all is going to fetch all of the sms logs in our database. Suppose there’s 100 million of them. That’d be a very expensive query, taking all 100M rows from the DB and loading them into memory on the server. Instead, we could say to the database, “Hey, could you give us just the sms logs from John’s campaign?”. Suppose there’s 30k of those. Now we only have to load 30k records into memory on our server instead of 100M. Much better.
I felt bad about this mistake. I’ve been programming for eight years. Of course you don’t want to load 100M records into memory on your server. Duh. How could I get such a basic thing wrong?
A cognitive behavioral therapist would call this a dysfunctional thought. And they’d ask me to come up with a rational response to the thought. Well, here’s my attempt at a rational response.
Imagine that you had a pyramid of things a programmer knows. At the bottom are basic things like what the map function does. At the top are advanced things like what a monad is. In the middle are stuff in between.
I think that this base layer of the pyramid is very, very wide. There are a lot of these “basic” things to know. It’s to the point that even senior engineers are going to have a very non-trivial amount of gaps in this level of their pyramid. For example, here is a rough sketch of what I’d imagine a senior engineer’s pyramid would look like versus a junior engineer’s pyramid.
The senior engineer did a pretty nice job of filling out that bottom level, but there are still some notable gaps. There are still definitely going to be times when the senior engineer stumbles upon basic things that they don’t know.
And this is why I say that my thought was dysfunctional. Just because you stumble across something basic that you don’t know doesn’t mean you’re a bad developer. Even good developers have lots of gaps at the base of their pyramids. You could very well be a good developer who just so happened to stumble upon one of those gaps.
Tangent: I really like this pyramid analogy. I think it shows how you can’t really be learning level two stuff if you don’t already have the level one stuff. Notice how in my diagrams that when there’s a gap at level one there’s never anything above it. Well, technically I think reality is more jenga, where you don’t need a 100% perfect foundation to build on top of, but that there are consequences of having a shaky foundation.
I also want to comment on how both the senior and junior engineers started moving up to level two without first filling out the entirety of level one. I think this is appropriate.
It’s been my experience that all senior developers have their fair share of gaps at the bottom level. I can’t think of anyone I’ve met who is an exception to this. Can you?
A great example is Dan Abramov, the creator of Redux, and a member of the React Core team. In one of my favorite blog posts ever, Dan bravely shares with the world a bunch of basic things that he doesn’t know. Some of the things that stood out to me as especially eye opening are:
“Modern CSS. I don’t know Flexbox or Grid. Floats are my jam.”
“CORS. I dread these errors! I know I need to set up some headers to fix them but I’ve wasted hours here in the past.”
“Algorithms. The most you’ll get out of me is bubble sort and maybe quicksort on a good day. I can probably do simple graph traversing tasks if they’re tied to a particular practical problem. I understand the O(n) notation but my understanding isn’t much deeper than ‘don’t put loops inside loops’.”
In that spirit, I’d like to list out some of the basic things that, after programming for eight years, I personally still don’t know:
How joins work in SQL. I have a vague idea, but I have trouble thinking about them and visualizing them.
Database normalization is just about avoiding duplication, right? Why is that such a big deal? Is it a such big deal?
I often lose track of what this is.
Without googling, I couldn’t tell you what is cool about Node.js. And after a quick google, I still don’t really get it. I’d have to think harder about it.
Relative paths always screw with me.
I’m not great with git. I don’t really understand the tradeoffs of merging vs rebasing, and sometimes I get myself into a pickle.
With CSS, I’m the opposite of Dan. Flexbox and Grid make sense to me, but floats are never something I’ve been able to develop a solid grasp of. In using them I am either referencing an example from Stack Overflow, or I have to play around until it works.
Speaking of CSS, I always have to look up how to center things. CSS Tricks’ article is my goto. I remember one time I was getting paid to tutor someone. He asked me how to center stuff, and I had to spend time fumbling around referencing the CSS Tricks article as I tried to explain it.
DNS stuff. Whenever I set up a new website, dealing with DNS stuff is always a struggle. I’ve actually been paying $7/month for a while to serve static content via Heroku instead of moving to Netlify because I can’t figure out how to migrate over.
I have an intuitive sense that watchers are something to avoid in Vue if you can, but I can’t really explain why.
To be clear, I’m not just talking about a front end developer not knowing the basics of how compilers work. I’m talking about a front end developer having level one gaps in normal, everyday things like floats and flexbox. About having basic gaps in your day-to-day domain, not just outside of it. The gaps will of course be less frequent when you’re inside your normal domain, but they are still very non-trivial.
I think I’ll stop here. I can go on of course, but this should get the point across.
Why write this post? These points all feel sorta obvious. Of course you can’t expect someone to know 100% of the basic things. Duh. No one is perfect.
It’s not that I expect people to disagree with the core point, but I do think that it’s something that is easy to lose sight of. And underestimate. So if you were previously in either of those boats, hopefully I’ve been able to bring you back to shore. And if you started off on shore, well, hopefully you’ve had a good time singing kumbaya with me.
Don’t feel bad about not knowing basic things
I recently made a mistake where I tried doing something like this in Ruby on Rails:
I did have a sense that it wasn’t The Rails Way, but I also had a sense that it was quite readable and sufficient. The code was working well locally, so I issued a pull request.
When I issued the PR, someone pointed out a major issue.
SmsLog.allis going to fetch all of the sms logs in our database. Suppose there’s 100 million of them. That’d be a very expensive query, taking all 100M rows from the DB and loading them into memory on the server. Instead, we could say to the database, “Hey, could you give us just the sms logs from John’s campaign?”. Suppose there’s 30k of those. Now we only have to load 30k records into memory on our server instead of 100M. Much better.I felt bad about this mistake. I’ve been programming for eight years. Of course you don’t want to load 100M records into memory on your server. Duh. How could I get such a basic thing wrong?
A cognitive behavioral therapist would call this a dysfunctional thought. And they’d ask me to come up with a rational response to the thought. Well, here’s my attempt at a rational response.
Imagine that you had a pyramid of things a programmer knows. At the bottom are basic things like what the
mapfunction does. At the top are advanced things like what a monad is. In the middle are stuff in between.I think that this base layer of the pyramid is very, very wide. There are a lot of these “basic” things to know. It’s to the point that even senior engineers are going to have a very non-trivial amount of gaps in this level of their pyramid. For example, here is a rough sketch of what I’d imagine a senior engineer’s pyramid would look like versus a junior engineer’s pyramid.
The senior engineer did a pretty nice job of filling out that bottom level, but there are still some notable gaps. There are still definitely going to be times when the senior engineer stumbles upon basic things that they don’t know.
And this is why I say that my thought was dysfunctional. Just because you stumble across something basic that you don’t know doesn’t mean you’re a bad developer. Even good developers have lots of gaps at the base of their pyramids. You could very well be a good developer who just so happened to stumble upon one of those gaps.
It’s been my experience that all senior developers have their fair share of gaps at the bottom level. I can’t think of anyone I’ve met who is an exception to this. Can you?
A great example is Dan Abramov, the creator of Redux, and a member of the React Core team. In one of my favorite blog posts ever, Dan bravely shares with the world a bunch of basic things that he doesn’t know. Some of the things that stood out to me as especially eye opening are:
“Modern CSS. I don’t know Flexbox or Grid. Floats are my jam.”
“CORS. I dread these errors! I know I need to set up some headers to fix them but I’ve wasted hours here in the past.”
“Algorithms. The most you’ll get out of me is bubble sort and maybe quicksort on a good day. I can probably do simple graph traversing tasks if they’re tied to a particular practical problem. I understand the O(n) notation but my understanding isn’t much deeper than ‘don’t put loops inside loops’.”
In that spirit, I’d like to list out some of the basic things that, after programming for eight years, I personally still don’t know:
How joins work in SQL. I have a vague idea, but I have trouble thinking about them and visualizing them.
Database normalization is just about avoiding duplication, right? Why is that such a big deal? Is it a such big deal?
I often lose track of what
thisis.Without googling, I couldn’t tell you what is cool about Node.js. And after a quick google, I still don’t really get it. I’d have to think harder about it.
Relative paths always screw with me.
I’m not great with git. I don’t really understand the tradeoffs of merging vs rebasing, and sometimes I get myself into a pickle.
With CSS, I’m the opposite of Dan. Flexbox and Grid make sense to me, but floats are never something I’ve been able to develop a solid grasp of. In using them I am either referencing an example from Stack Overflow, or I have to play around until it works.
Speaking of CSS, I always have to look up how to center things. CSS Tricks’ article is my goto. I remember one time I was getting paid to tutor someone. He asked me how to center stuff, and I had to spend time fumbling around referencing the CSS Tricks article as I tried to explain it.
DNS stuff. Whenever I set up a new website, dealing with DNS stuff is always a struggle. I’ve actually been paying $7/month for a while to serve static content via Heroku instead of moving to Netlify because I can’t figure out how to migrate over.
I have an intuitive sense that watchers are something to avoid in Vue if you can, but I can’t really explain why.
To be clear, I’m not just talking about a front end developer not knowing the basics of how compilers work. I’m talking about a front end developer having level one gaps in normal, everyday things like floats and flexbox. About having basic gaps in your day-to-day domain, not just outside of it. The gaps will of course be less frequent when you’re inside your normal domain, but they are still very non-trivial.
I think I’ll stop here. I can go on of course, but this should get the point across.
Why write this post? These points all feel sorta obvious. Of course you can’t expect someone to know 100% of the basic things. Duh. No one is perfect.
It’s not that I expect people to disagree with the core point, but I do think that it’s something that is easy to lose sight of. And underestimate. So if you were previously in either of those boats, hopefully I’ve been able to bring you back to shore. And if you started off on shore, well, hopefully you’ve had a good time singing kumbaya with me.