Back of the envelope statistical significance calculations: “Yes, sales per lead have gone up since the change went out, but we’ve only had 150 sales come through since then so we’d expect a difference of about 12 just based on random noise, and indeed the difference is about 10 sales.”
Noticing unknown unknowns: “We’d expect hourly signups to vary by about 50 people based on independent random decisions, but we’re actually seeing variation of about 250 people, without any clear time-of-day pattern. There must be some underlying factor which makes a bunch who come in around the same time all more/less likely to sign up.” (In this case, the underlying factor was that our server wasn’t able to handle the load, so when it got behind lots of people had lots of lag at the same time.)
Noticing corner cases: “If two people are sharing one account, what do we do when they both edit at the same time? What if someone updates X, but we’ve already done a bunch of logic using the original X value, and now we need them to go back and input some other information?”
Critical path reasoning: “We usually end up waiting around on the appraisal—that’s what takes longest. And we’re not ordering the appraisal until we have form X. But we don’t actually need to wait that long—we can parallelize, and order the appraisal while we’re still waiting on form X. We could also parallelize fetching the insurance forms, but that won’t matter much—they’re usually pretty fast anyway.”
Sparsity: “This loop is running over every possible pair of words. That’s something like 100M word pairs. But the data only contains 100k sentences with ~10 word pairs in each of them, so at most only about 1M word pairs actually occur. If we loop over sentences and only look at pairs which actually occur, that should be at least a 100X speedup.”
Little-o reasoning: “These changes just aren’t that big, so the relationship should be roughly linear.”
Queueing theory: “If we’re trying to keep all of our people busy all of the time, then our wait times are going to grow longer and longer. If we want short wait times, then we need to have idle capacity.”
I’m excluding economic-style reasoning, because so much has already been written about applications of economics in everyday life. But if you want me to add some econ examples, let me know.
Back of the envelope statistical significance calculations: “Yes, sales per lead have gone up since the change went out, but we’ve only had 150 sales come through since then so we’d expect a difference of about 12 just based on random noise, and indeed the difference is about 10 sales.”
Noticing unknown unknowns: “We’d expect hourly signups to vary by about 50 people based on independent random decisions, but we’re actually seeing variation of about 250 people, without any clear time-of-day pattern. There must be some underlying factor which makes a bunch who come in around the same time all more/less likely to sign up.” (In this case, the underlying factor was that our server wasn’t able to handle the load, so when it got behind lots of people had lots of lag at the same time.)
Noticing corner cases: “If two people are sharing one account, what do we do when they both edit at the same time? What if someone updates X, but we’ve already done a bunch of logic using the original X value, and now we need them to go back and input some other information?”
Critical path reasoning: “We usually end up waiting around on the appraisal—that’s what takes longest. And we’re not ordering the appraisal until we have form X. But we don’t actually need to wait that long—we can parallelize, and order the appraisal while we’re still waiting on form X. We could also parallelize fetching the insurance forms, but that won’t matter much—they’re usually pretty fast anyway.”
Sparsity: “This loop is running over every possible pair of words. That’s something like 100M word pairs. But the data only contains 100k sentences with ~10 word pairs in each of them, so at most only about 1M word pairs actually occur. If we loop over sentences and only look at pairs which actually occur, that should be at least a 100X speedup.”
Little-o reasoning: “These changes just aren’t that big, so the relationship should be roughly linear.”
Queueing theory: “If we’re trying to keep all of our people busy all of the time, then our wait times are going to grow longer and longer. If we want short wait times, then we need to have idle capacity.”
I’m excluding economic-style reasoning, because so much has already been written about applications of economics in everyday life. But if you want me to add some econ examples, let me know.