I’d also add: 5) engineers often totally forget about the black swans (or unknown unknowns) - especially obscure edge-cases that require special handling.
I’m not sure that’s what is meant by a black swan, but in any case, this would be a factor only if managers more accurately predict the impact of these events. (If it’s a case of known-but-unusual things happening in part of the problemspace, that’s an issue of defining the problem appropriately for the engineers, and can be corrected before the solution is implemented without a higher time-discount or margin of safety.)
I agree that ‘black swan’ isn’t really appropriate here, but can you clarify what you mean by ‘that’s an issue of defining the problem appropriately for the engineers’?
I just mean that if you don’t tell the engineers that special constraints apply in part of the designspace that render any design in that part horrible, it’s not their fault if, not knowing this extra constraint, they offer that as their solution because it is otherwise optimal.
Yes, that’s true. and we should make sure that a client thinks deeply about all the cases that would reasonably come up.
...but I often find that there are situations that come up that really could not have been predicted in advance.
A small example for an e-commerce site might be the effect that adding multiple currencies has on dealing with “$X off” promotions if all you do is scale from a single currency you end up with $5 off or £3.24 off… which is ugly… and unfortunately required us to rework promotions to allow us to add “currency value overrides” for this kind of promotion.
It was unexpected, we didn’t know about it in advance( either us or our clients) but it was obvious-in-hindsight that it would be unacceptable to the client/customers.
I agree with you in that respect. If neither managers nor engineers are aware of special conditions that attach in part of the designspace until it catastrophically happens, then that is a black swan, and a reason to temper judgment with a higher discount rate.
I was only adding that if engineers come back with a solution that, given what they were told, is optimal, when the managers knew all along about special conditions make the proposed solution unfeasilbe, it’s not a black swan, nor really fair to say, “silly engineers—not accounting for the real world!” (It’s at that point that you should re-iterate, adding that constraint to the spec this time and you certainly shouldn’t go forward with it.) And that, if this is the problem, it’s not the kind of thing that requires a higher discount rate or margin of safety, just better problem specification.
Definitely agree with you on that one. I’d guess it more a problem of scope-creep.
You can also solve that by discounting, but it’s much better to solve by enforcing different accounting for change-requests. I’ve found that if you keep time spent on scope-creep as a separate bottom-line, it draws more attention to the problem (which makes it more likely to get solved).
It’s almost certainly a reference to this theory, which essentially describes extremely difficult to predict events that have a disproportionate impact. Unfortunately the very nature of these events makes them difficult to control for.
I’m familiar with the term, and it’s not the same thing as obscure edge cases that require special handling; rather, it’s unforseeable events that have disproportionately large impact, nothing to do with being an edge case or requiring special handling per se. An unknown unknown that never has an impact is not a black swan. And something that requires special handling isn’t a black swan if that “special handling” is known in advance (even if the fact that it will happen is not).
I think an unforeseeable edge case or bug that requires deep refactoring and severely cuts into allotted development time fits the bill for a black swan dead on.
I’m not sure that’s what is meant by a black swan, but in any case, this would be a factor only if managers more accurately predict the impact of these events. (If it’s a case of known-but-unusual things happening in part of the problemspace, that’s an issue of defining the problem appropriately for the engineers, and can be corrected before the solution is implemented without a higher time-discount or margin of safety.)
Yes, perhaps that should instead read:
5) engineers often totally forget about the unknown unknowns or even black swans.
I agree that ‘black swan’ isn’t really appropriate here, but can you clarify what you mean by ‘that’s an issue of defining the problem appropriately for the engineers’?
I just mean that if you don’t tell the engineers that special constraints apply in part of the designspace that render any design in that part horrible, it’s not their fault if, not knowing this extra constraint, they offer that as their solution because it is otherwise optimal.
Yes, that’s true. and we should make sure that a client thinks deeply about all the cases that would reasonably come up.
...but I often find that there are situations that come up that really could not have been predicted in advance.
A small example for an e-commerce site might be the effect that adding multiple currencies has on dealing with “$X off” promotions if all you do is scale from a single currency you end up with $5 off or £3.24 off… which is ugly… and unfortunately required us to rework promotions to allow us to add “currency value overrides” for this kind of promotion.
It was unexpected, we didn’t know about it in advance( either us or our clients) but it was obvious-in-hindsight that it would be unacceptable to the client/customers.
I agree with you in that respect. If neither managers nor engineers are aware of special conditions that attach in part of the designspace until it catastrophically happens, then that is a black swan, and a reason to temper judgment with a higher discount rate.
I was only adding that if engineers come back with a solution that, given what they were told, is optimal, when the managers knew all along about special conditions make the proposed solution unfeasilbe, it’s not a black swan, nor really fair to say, “silly engineers—not accounting for the real world!” (It’s at that point that you should re-iterate, adding that constraint to the spec this time and you certainly shouldn’t go forward with it.) And that, if this is the problem, it’s not the kind of thing that requires a higher discount rate or margin of safety, just better problem specification.
Definitely agree with you on that one. I’d guess it more a problem of scope-creep. You can also solve that by discounting, but it’s much better to solve by enforcing different accounting for change-requests. I’ve found that if you keep time spent on scope-creep as a separate bottom-line, it draws more attention to the problem (which makes it more likely to get solved).
It’s almost certainly a reference to this theory, which essentially describes extremely difficult to predict events that have a disproportionate impact. Unfortunately the very nature of these events makes them difficult to control for.
I’m familiar with the term, and it’s not the same thing as obscure edge cases that require special handling; rather, it’s unforseeable events that have disproportionately large impact, nothing to do with being an edge case or requiring special handling per se. An unknown unknown that never has an impact is not a black swan. And something that requires special handling isn’t a black swan if that “special handling” is known in advance (even if the fact that it will happen is not).
I think an unforeseeable edge case or bug that requires deep refactoring and severely cuts into allotted development time fits the bill for a black swan dead on.
Understood. For some reason I misread your original post as saying that you weren’t sure what a black swan was. I agree with your analysis here.