When I was in graduate school, they let you take the qualifying exams twice. So I didn’t study at all the first time, confident that if I failed, I could study and pass the second time. So in that spirit, here’s me answering the questions without having read the book, and without Googling anything.
1. Learning is a change in your mental model; validated learning is a change in your model that is either tied to a core metric or that you then confirm through tests.
2. I’m going to guess True. That is, both of those metrics (revenue and number of customers) are pretty hard to fake and so feel validated, and exponential growth means ‘something real is happening’ in startup land. It feels a bit like it’s a trick question, because the metrics are between clearly real/good metrics (like profit) and more fake metrics (like number of users, as opposed to number of active users), but my guess is these are real according to the author.
3. Hmmm. “by forcing you to iterate quickly, and by keeping you in contact with users / learning from reality.”
4. Innovation accounting presumably has something more like “net present value” as calculated from projections; I’m going to guess this means stuff like tracking growth rate instead of current level (since the number of users will still look explosive even if your growth is dropping from 5% to 4% to 3%).
5. Push has stations doing work whenever they get the inputs and then sending it on; so maybe a supplier makes screws and just sends a hundred box of screws a week or w/e. The engine station makes engines and then sends them to assembly, assembly starts at the start and pushes things down the line, etc.; basically you have lots of work-in-progress (WIP), fungible workers working on the wrong thing (someone who could assemble doors and windows is just doing whatever they’re assigned, which might not be the important thing), and potentially imbalanced flows / stations starved of necessary inputs. Pull instead flows backwards from the outputs. I’m going to describe a particular way to implement this to make it concrete, but there are lots of different ways to do this / this is only sometimes applicable. You have a number of cars you want to get out the door, and so you look at the final station (polishing, say), and have a sense of what rate it can polish at that determines how many cars needing polishing should be sitting in the ‘in box’ for that station. Whenever the in box drops below that, the previous station (upholstery, say) gets an order, which it then tries to fulfill, which maybe drops some of its inboxes below their level, which then makes previous stations generate more (screws, seats, etc.). A thing that’s nice about pull is that you’ve put a control system on the WIP, hoping to make sure that everyone is able to do the work that’s most useful at the moment. If you don’t actually need any more screws, you don’t make any more screws. If you have a thousand different parts, the WIP control system is less good, and instead you just want to send orders directly to all the stations, tho prioritization is then more of a challenge.
6. Presumably the pull is something like “growth”; like, you have whatever core metric you care about (like % increase in revenue week-on-week), and then you try to reason backwards from that to everything else the company is doing. You don’t have an engineer who just comes in and cleans up the code every day and then goes home (a more push-based model), you have a story that goes from % increase in revenue to shipping new features to what the engineer needs to be doing this week (which might be cleaning up the codebase so that you can make new features).
7. True in that you’re letting the system plan for you, instead of needing your human planners to be forecasting demand correctly. But obviously the WIP cost is a function of the underlying uncertainty.
8. False; lean often points you towards flexibility instead of rigidity, and rigidity is baked into a lot of ‘economies of scale’ things. Instead of getting a deal on 10,000 screws, you buy them in boxes of 100 whenever you open one and only have five boxes on hand. This helps when you suddenly stop needing as many screws, and also if you suddenly need lots of screws (since you can easily buy more boxes, where it may be difficult to shift the delivery date on your huge crate of screws).
9. First, the dad is able to ship his first letter sooner. Second, the dad learns things from going through the whole cycle; if, for example, the fold of the first letter doesn’t quite fit in the envelope, he can improve the fold for next time, whereas the kids will discover that all of their letters don’t quite fit in the envelope.
10. True. Employees will spend more time switching, which drops productivity, and maybe even waiting, which drops productivity. This is the cost of flexibility, and ends up paying for itself because the increased prioritization means they’re getting less output out the door but the output is more valuable.
11. Hmmm, I don’t think I’ve heard this phrase before, but I assume it means something like trying to do lots of things at once (like the kids doing the letters in an assembly-line way without feedback), such that the product is late and low-quality, and in particular having to abandon lots of WIP when the market changes underneath you. “Well, we didn’t send all of the letters in time for Christmas, and now we have to start our Valentine’s letters, which really can’t use much of the WIP we have lying around from Christmas.”
12. It’s a negative feedback system for faults, errors, and incidents. Something goes wrong, and you try to get information from that event fed back into five generating systems (as defined by levels of abstraction). This then drives down (you hope) the number of future errors, eventually controlling it to 0 (in the absence of changes that then introduce new faults).
13. Hmm, I can see two meanings here. The first one, that I’m more confident in, is the “any worker can halt the line at any time” system, where giving anyone the power to identify problems immediately means that you are always either 1) producing good product or 2) fixing the line such that it produces good product. “Production” consists of 1 and 2 together, and not of 3) producing bad product, since the outputs of 3 will just have to be thrown away. The other meaning is that if your station doesn’t have any needed output, you shouldn’t do something just to not be idle; this is so that if a needed order does come in, you can immediately start on it so that it’s done as soon as possible.
When I was in graduate school, they let you take the qualifying exams twice. So I didn’t study at all the first time, confident that if I failed, I could study and pass the second time. So in that spirit, here’s me answering the questions without having read the book, and without Googling anything.
1. Learning is a change in your mental model; validated learning is a change in your model that is either tied to a core metric or that you then confirm through tests.
2. I’m going to guess True. That is, both of those metrics (revenue and number of customers) are pretty hard to fake and so feel validated, and exponential growth means ‘something real is happening’ in startup land. It feels a bit like it’s a trick question, because the metrics are between clearly real/good metrics (like profit) and more fake metrics (like number of users, as opposed to number of active users), but my guess is these are real according to the author.
3. Hmmm. “by forcing you to iterate quickly, and by keeping you in contact with users / learning from reality.”
4. Innovation accounting presumably has something more like “net present value” as calculated from projections; I’m going to guess this means stuff like tracking growth rate instead of current level (since the number of users will still look explosive even if your growth is dropping from 5% to 4% to 3%).
5. Push has stations doing work whenever they get the inputs and then sending it on; so maybe a supplier makes screws and just sends a hundred box of screws a week or w/e. The engine station makes engines and then sends them to assembly, assembly starts at the start and pushes things down the line, etc.; basically you have lots of work-in-progress (WIP), fungible workers working on the wrong thing (someone who could assemble doors and windows is just doing whatever they’re assigned, which might not be the important thing), and potentially imbalanced flows / stations starved of necessary inputs.
Pull instead flows backwards from the outputs. I’m going to describe a particular way to implement this to make it concrete, but there are lots of different ways to do this / this is only sometimes applicable. You have a number of cars you want to get out the door, and so you look at the final station (polishing, say), and have a sense of what rate it can polish at that determines how many cars needing polishing should be sitting in the ‘in box’ for that station. Whenever the in box drops below that, the previous station (upholstery, say) gets an order, which it then tries to fulfill, which maybe drops some of its inboxes below their level, which then makes previous stations generate more (screws, seats, etc.).
A thing that’s nice about pull is that you’ve put a control system on the WIP, hoping to make sure that everyone is able to do the work that’s most useful at the moment. If you don’t actually need any more screws, you don’t make any more screws. If you have a thousand different parts, the WIP control system is less good, and instead you just want to send orders directly to all the stations, tho prioritization is then more of a challenge.
6. Presumably the pull is something like “growth”; like, you have whatever core metric you care about (like % increase in revenue week-on-week), and then you try to reason backwards from that to everything else the company is doing. You don’t have an engineer who just comes in and cleans up the code every day and then goes home (a more push-based model), you have a story that goes from % increase in revenue to shipping new features to what the engineer needs to be doing this week (which might be cleaning up the codebase so that you can make new features).
7. True in that you’re letting the system plan for you, instead of needing your human planners to be forecasting demand correctly. But obviously the WIP cost is a function of the underlying uncertainty.
8. False; lean often points you towards flexibility instead of rigidity, and rigidity is baked into a lot of ‘economies of scale’ things. Instead of getting a deal on 10,000 screws, you buy them in boxes of 100 whenever you open one and only have five boxes on hand. This helps when you suddenly stop needing as many screws, and also if you suddenly need lots of screws (since you can easily buy more boxes, where it may be difficult to shift the delivery date on your huge crate of screws).
9. First, the dad is able to ship his first letter sooner. Second, the dad learns things from going through the whole cycle; if, for example, the fold of the first letter doesn’t quite fit in the envelope, he can improve the fold for next time, whereas the kids will discover that all of their letters don’t quite fit in the envelope.
10. True. Employees will spend more time switching, which drops productivity, and maybe even waiting, which drops productivity. This is the cost of flexibility, and ends up paying for itself because the increased prioritization means they’re getting less output out the door but the output is more valuable.
11. Hmmm, I don’t think I’ve heard this phrase before, but I assume it means something like trying to do lots of things at once (like the kids doing the letters in an assembly-line way without feedback), such that the product is late and low-quality, and in particular having to abandon lots of WIP when the market changes underneath you. “Well, we didn’t send all of the letters in time for Christmas, and now we have to start our Valentine’s letters, which really can’t use much of the WIP we have lying around from Christmas.”
12. It’s a negative feedback system for faults, errors, and incidents. Something goes wrong, and you try to get information from that event fed back into five generating systems (as defined by levels of abstraction). This then drives down (you hope) the number of future errors, eventually controlling it to 0 (in the absence of changes that then introduce new faults).
13. Hmm, I can see two meanings here. The first one, that I’m more confident in, is the “any worker can halt the line at any time” system, where giving anyone the power to identify problems immediately means that you are always either 1) producing good product or 2) fixing the line such that it produces good product. “Production” consists of 1 and 2 together, and not of 3) producing bad product, since the outputs of 3 will just have to be thrown away.
The other meaning is that if your station doesn’t have any needed output, you shouldn’t do something just to not be idle; this is so that if a needed order does come in, you can immediately start on it so that it’s done as soon as possible.