I am continually amazed at the amount of data that should be put and manipulated in a proper database that’s mucked about in spreadsheets instead.
I had a job at an actuarial firm once and it would frustrate the hell out of me to see people constantly trying to do joins and other relational operations using Excel formulae, not knowing that that’s what they were doing, spending ages and ages on such tasks, holding Excel training sessions on how to use functions like INDEX and MATCH, using a bunch of custom-made macros to give names to columns and fill down formulas, and still making tons of mistakes that wouldn’t even be possible with a SQL query.
They were bleeding time and money at an astronomical rate, but it didn’t matter because their competitors were just as bad.
Working with a ‘proper’ database is a skill I still lack, which I would very much like to acquire. I was hoping this job would help, but instead I just enter everything into Excel. (To be fair, you can manipulate data a fair bit using Excel formulas. I just don’t know all of them.)
Try MS Access. It is pretty simple to use, and has a nice export to Excel feature. Once you “get” Access it should be easy to move on to more advanced DB tools. Access also has a better UI that should make data-input work easier.
I agree in some sense, but disagree in another. I am fast at Excel. I don’t need to use to mouse, look at the menus, or pause to find anything I’m looking for, because I’ve internalized the keyboard shortcuts and created quick macros for the things I need to do. People get a little flustered when they see me work in Excel because it looks like it’s magically doing stuff, but it just comes from lots and lots of repetition.
Contrast this with a proper database, where I need to figure out some way to load the data in, make sure my query accounts for every place there might be a null value, then make some change that might break my previous queries if I need to add or change a column or something. And then if I need to take a slice of data and present it, I have to load it into Excel anyway.
For very large datasets with fairly static requirements, I use a database as is proper. But for anything less than 100K rows, give me a spreadsheet any day.
I had a job at an actuarial firm once and it would frustrate the hell out of me to see people constantly having to spend ages and ages doing ‘join’ operations using Excel formulae, not knowing that that’s what they were doing, spending ages and ages on such tasks, holding Excel training sessions on how to use functions like INDEX and MATCH, using a bunch of custom-made macros just to give names to columns, and making mistakes that wouldn’t even be possible with a SQL query.
They were bleeding time and money at an astronomical rate, but it didn’t matter because their competitors were just as bad.
A few thoughts:
I am continually amazed at the amount of data that should be put and manipulated in a proper database that’s mucked about in spreadsheets instead.
Gene Callahan at ThinkMarkets recently had a post on how health care is especially bad when it comes to technology.
Eric Falkenstein says there are a lot of “skilled” jobs that could be replaced by computers too.
I had a job at an actuarial firm once and it would frustrate the hell out of me to see people constantly trying to do joins and other relational operations using Excel formulae, not knowing that that’s what they were doing, spending ages and ages on such tasks, holding Excel training sessions on how to use functions like INDEX and MATCH, using a bunch of custom-made macros to give names to columns and fill down formulas, and still making tons of mistakes that wouldn’t even be possible with a SQL query.
They were bleeding time and money at an astronomical rate, but it didn’t matter because their competitors were just as bad.
Excel is mostly used as a single-table database system. It has the important virtue of being an improvement on nothing.
Working with a ‘proper’ database is a skill I still lack, which I would very much like to acquire. I was hoping this job would help, but instead I just enter everything into Excel. (To be fair, you can manipulate data a fair bit using Excel formulas. I just don’t know all of them.)
Try MS Access. It is pretty simple to use, and has a nice export to Excel feature. Once you “get” Access it should be easy to move on to more advanced DB tools. Access also has a better UI that should make data-input work easier.
Thanks.
I agree in some sense, but disagree in another. I am fast at Excel. I don’t need to use to mouse, look at the menus, or pause to find anything I’m looking for, because I’ve internalized the keyboard shortcuts and created quick macros for the things I need to do. People get a little flustered when they see me work in Excel because it looks like it’s magically doing stuff, but it just comes from lots and lots of repetition.
Contrast this with a proper database, where I need to figure out some way to load the data in, make sure my query accounts for every place there might be a null value, then make some change that might break my previous queries if I need to add or change a column or something. And then if I need to take a slice of data and present it, I have to load it into Excel anyway.
For very large datasets with fairly static requirements, I use a database as is proper. But for anything less than 100K rows, give me a spreadsheet any day.
I had a job at an actuarial firm once and it would frustrate the hell out of me to see people constantly having to spend ages and ages doing ‘join’ operations using Excel formulae, not knowing that that’s what they were doing, spending ages and ages on such tasks, holding Excel training sessions on how to use functions like INDEX and MATCH, using a bunch of custom-made macros just to give names to columns, and making mistakes that wouldn’t even be possible with a SQL query.
They were bleeding time and money at an astronomical rate, but it didn’t matter because their competitors were just as bad.