Two years ago I decided I was going to start running between my house and the subway station each day on my commute, for a total of about five miles a week. Initially I was very interested in getting faster, and would time myself a couple days a week, trying to beat my previous best:
Unfortunately, after six months I was pushing myself too hard, and my knees started hurting. I stopped timing my runs and stopped running so hard, and they got better again. I’m still running, but at a gentler pace.
This experience was a good illustration of how optimizing for a metric can often bring you towards your overall goals for a while, but then continuing to optimize on it can start to bring you away from them again.
It’s a good illustration if you’re optimizing by pushing one variable – running harder. It’s not a good illustration in general. Consider my case, which is analogous to yours:
My overall goals are health, fitness and wellbeing (as I assume yours is). And I lift weights, for example doing Turkish Get-Ups with a kettlebell. I started this with no weights, then increased to 12 kg, 16 kg, 20 kg. So my metric is weight x sets. Whenever I increased weight, I got injuries/pains. – First an irritated muscle under my shoulder blade, then pains around my thoracic spine, then a muscle on the outside of my shoulder that felt like it was getting pulled.
I could have said that I’m pushing myself too hard and decided to stay at the old weight. Instead I turned other knobs: I improved my warm-up, did some mobilizations and fixed my form, even booking a personal trainer/physiotherapist a few times. Similar things happened with every weighted exercise I’m doing. It’s just hard to move correctly. Increasing the load (weight, speed etc.) exposes your faults. Then you fix them.
So optimizing my metrics brought me towards my overall goals, continuing to optimize started to bring me away from them, but still continuing to optimize (by taking different actions) brought me even closer to them: greater load plus more correct movements are part of health, fitness, wellbeing.
I haven’t had the OP’s problem, but I set goals that was more fixed (X speed, Y distance), and just aimed to be fast enough.
There were two bus stops (let’s call them A and B) that weren’t too far apart. If on my way to either of them, I noticed the bus going from A to B, I started running to catch up. I also ran other times so I’d get places faster. The goal wasn’t ‘go as fast as I can to beat my previous time’ but ‘get places faster’. If you run too fast (for where you’re at physically), then you have to slow down, and don’t arrive as quickly as you would’ve if you’d paced yourself more. Also, while you’re recovering, you’re less able to do other things, and if the goal is decreasing ‘travel time + recovery time’ then that can mean not pushing yourself too much more (until you improve) once you hit that sweet spot.
Venkat talks about this in his mediocrity sequence, that kakonomics are a kind of defense against goodharting yourself in the tails since tails diverge.