Maybe the whole moral mazes theory is false and this is just BS.
When you have a reasonable feedback loop, the costs imposed here add up to a “significant diminishing marginal return”, but maybe don’t actually turn negative. “Big organizations have diminishing returns to more labor” was already part of my model before Moral Mazes, and Moral Mazes just happen to be one of the mechanisms.
Hypothesis 1: innovation mostly happens in startups, and then they eventually get bought or copied by big corporations. (I think this hypothesis is actually false in many ways – my understanding is that serious research R&D type innovation tends to happen in huge companies like IBM, Google, etc, and startups do something more like “combining obvious ideas in simple ways.” But, still seems at least somewhat relevant)
evidence for Hypothesis 1: DeepMind was created originally as a small lab, that was later bought by google.
evidence against: Google seems to have other reseearch departments doing at least somewhat comparable research (but, I don’t actually know offhand where Google Brain came from, maybe it was also a small lab that got bought?)
Hypothesis 2: The innovation we see in big companies comes from R&D departments that are somehow particularly-sheltered from goodhart dynamics. (Maybe a big company has lots of random pockets of organization within it, which vary in maziness, there’s just some selection effects on where innovation happens)
My understanding is that Amazon-in-particular has a fairly parallelized structure, which seems less vulnerable to hierarchical-goodhart dynamics. Someone in the comments here noted that Walmart might be reasonable maze resistant by “lots of the employees work in a place where the object level work is right in front of them”, which also seems reasonably parallelized. (Amazon and Walmart are the two largest companies by employee-count.
Some progress comes from incremental engineering. Some comes from deeper research. When I about my day-to-day working at LessWrong, there is just a lot of obvious stuff to work on / improve.
Experiment to run: Check for the highest rated progress (on Amazon? On some kind of consumer reports? On wirecutter?). See which companies produced the best product. See how big the companies that produce the best products are. (I think operationalizing the prediction here is a bit tricky, requires getting good data, dealing with some confounders and stuff. Will think on it)
Google Brain was developed as part of X (Google’s “moonshot factory”), which is their way of trying to create startups/startup culture within a large corporation. So was Waymo.
I agree that’s confusing. Some thoughts.
Maybe the whole moral mazes theory is false and this is just BS.
When you have a reasonable feedback loop, the costs imposed here add up to a “significant diminishing marginal return”, but maybe don’t actually turn negative. “Big organizations have diminishing returns to more labor” was already part of my model before Moral Mazes, and Moral Mazes just happen to be one of the mechanisms.
Hypothesis 1: innovation mostly happens in startups, and then they eventually get bought or copied by big corporations. (I think this hypothesis is actually false in many ways – my understanding is that serious research R&D type innovation tends to happen in huge companies like IBM, Google, etc, and startups do something more like “combining obvious ideas in simple ways.” But, still seems at least somewhat relevant)
evidence for Hypothesis 1: DeepMind was created originally as a small lab, that was later bought by google.
evidence against: Google seems to have other reseearch departments doing at least somewhat comparable research (but, I don’t actually know offhand where Google Brain came from, maybe it was also a small lab that got bought?)
Hypothesis 2: The innovation we see in big companies comes from R&D departments that are somehow particularly-sheltered from goodhart dynamics. (Maybe a big company has lots of random pockets of organization within it, which vary in maziness, there’s just some selection effects on where innovation happens)
My understanding is that Amazon-in-particular has a fairly parallelized structure, which seems less vulnerable to hierarchical-goodhart dynamics. Someone in the comments here noted that Walmart might be reasonable maze resistant by “lots of the employees work in a place where the object level work is right in front of them”, which also seems reasonably parallelized. (Amazon and Walmart are the two largest companies by employee-count.
Some progress comes from incremental engineering. Some comes from deeper research. When I about my day-to-day working at LessWrong, there is just a lot of obvious stuff to work on / improve.
Experiment to run: Check for the highest rated progress (on Amazon? On some kind of consumer reports? On wirecutter?). See which companies produced the best product. See how big the companies that produce the best products are. (I think operationalizing the prediction here is a bit tricky, requires getting good data, dealing with some confounders and stuff. Will think on it)
Google Brain was developed as part of X (Google’s “moonshot factory”), which is their way of trying to create startups/startup culture within a large corporation. So was Waymo.