I think a lot of the lazy engineering you point at is a rational allocation of resources, and not a problem with knowledge. Existing software engineers could easily optimize our existing software if they tried, but rewriting Electron apps as native code trades scarce resources (engineer time) for abundant resources (compute), and people don’t do it for the same reason cooks don’t try to optimize the amount of water they use when cooking.
I think there’s an economy-wide version of what you’re talking about though, where software engineering draws ~all of the knowledge workers and money, so we under-invest in things like biology, materials science, medicine, power, etc.
I think there’s an economy-wide version of what you’re talking about though, where software engineering draws ~all of the knowledge workers and money, so we under-invest in things like biology, materials science, medicine, power, etc.
1) Tradeoffs that seem reasonable at the moment may appear less reasonable later, when the environment changes. For example, when web pages were mostly texts, it didn’t matter if there was some bloat in the web browser. But then the web pages themselves exploded in size, and now opening five new tabs of Less Wrong caused my Firefox to slow down and sometimes crash, which makes me angry at both Less Wrong and Firefox—I see no reason why inactive tabs should tax the computer so much. So today, making the browsers more efficient would help all Less Wrong readers, all Substack readers, all Electron app users, etc.
2) The usual problem that creating value is not the same as capturing value, and the companies only care about the latter. Making each part of the system more efficient makes the entire system more efficient as a whole, but people are only willing to pay for some parts of the system.
For the browser issue your anger should be directed entirely at LessWrong[1]. Browsers are one of the few types of software which are still hyper-optimized, but there’s only so much you can do when people decide to render a complex application client-side using 6 MB of JavaScript:
Inactive tabs use memory because the alternative is losing your state when you switch tabs and having visible lag and reseting the state when you switch back. It’s possible browser makers are taking the wrong tradeoff and users don’t want tabs to remain in memory, but they’re not being inefficient, they’re optimizing for UX over memory.
Similarly, Electron is highly optimized for what it is; there’s just limits to how efficient you can make a browser while still presenting a fully standards-compliant JavaScript and DOM implementation. Also Electron can’t stop people from writing inefficient JavaScript.
Assuming you think the LessWrong team should stop doing some of the other things they’re doing and optimize performance instead. I disagree but this is a value judgement.
“Existing software engineers could easily optimize our existing software”
I don’t think they actually could! Even if software engineers really wanted to (which they generally don’t) and had the skills to (which seems to be becoming rarer), the software belongs to corporations, not to the engineers, and the corporations would never let their engineers optimise their software like this. (And if they did, they’d switfly be outcompeted by other corporations that could ship faster more featureful software.)
I think “ability to write efficient, optimised consumer software” is essentially no longer accessible to our civilisation for inescapable economic reasons, just like “ability to build beautiful architecture instead of featureless undecorated glass rectangles”, “ability to build interesting cars instead of bland blob-shaped automatic five-door huge-touchscreen front-wheel-drive SUVs”, etc. etc.)
“Electron apps as native code trades scarce resources (engineer time) for abundant resources (compute)”
I agree this tradeoff is definitely a factor—but I don’t think it’s the only tradeoff. We’re also trading-off things the average user doesn’t understand or notice (efficiency, privacy, reliability) for things they do notice (features, a fast release schedule, fancy graphics/UI, network-effects). That’s why the Microsoft Windows “start menu” is now a React app.
I think this results in worse software but that it’s inevitable and out of our control, not a choice where we could choose to do it differently if we wanted to: the corporations that control almost-all closed-source software would never let us optimise their software for things their users barely even noticed, even if it made the software better for those same users, and if by some miracle a corporation did let us they’d rapidly lose all their users to rival corps.
abundant resources (compute)
The resources are only abundant so long as we keep pounding away on the upgrade treadmill and never fall behind; today’s high-end phone or computer is tomorrow’s useless e-waste, and that’s a problem that should be (in theory!) entirely fixable in software. The reason a 5-year-old phone takes ten seconds to open a web browser, or a 5-year-old-video card can’t play a modern AAA computer game, or a 7 or 8 year old PC can’t even fit Microsoft Windows + Google Chrome into RAM is problems with the software, not with the hardware.
(Obligatory Linux/FLOSS mention: there does exist super-optimised software that has all the superficial features users notice and all the under-the-bonnet features that make the software actually good/effective to use and it’ll run snappily on a fifteen-year-old computer and it’s free. Why consumers don’t seem to want to touch it is a mystery to me!)
I’m not really sure if we disagree on the high level situation.
You mention that corporations won’t pay their engineers to optimize software, and that a corporation that does would be outcompeted because users prefer software with more features over optimized software, and you even note that in some cases better optimized software with fewer features is free and users still don’t choose it.
I think you’re overestimating how hard it is to write optimized native apps though. It’s tedious to write 5 native apps instead of one Electron app, and the frameworks are worse and slower to work with, but it’s not that hard (also if you need to display anything, you might need to embed a browser anyway). And I think AAA games are not a case of software being less optimized. I expect that on the low end, game developers are putting less effort into optimization, but AAA games on high settings don’t work on old video cards because new video cards are magic (real time ray-tracing!).
I think a lot of the lazy engineering you point at is a rational allocation of resources, and not a problem with knowledge. Existing software engineers could easily optimize our existing software if they tried, but rewriting Electron apps as native code trades scarce resources (engineer time) for abundant resources (compute), and people don’t do it for the same reason cooks don’t try to optimize the amount of water they use when cooking.
I think there’s an economy-wide version of what you’re talking about though, where software engineering draws ~all of the knowledge workers and money, so we under-invest in things like biology, materials science, medicine, power, etc.
Yes, to me looks like it’s happening a lot.
Two possible objections:
1) Tradeoffs that seem reasonable at the moment may appear less reasonable later, when the environment changes. For example, when web pages were mostly texts, it didn’t matter if there was some bloat in the web browser. But then the web pages themselves exploded in size, and now opening five new tabs of Less Wrong caused my Firefox to slow down and sometimes crash, which makes me angry at both Less Wrong and Firefox—I see no reason why inactive tabs should tax the computer so much. So today, making the browsers more efficient would help all Less Wrong readers, all Substack readers, all Electron app users, etc.
2) The usual problem that creating value is not the same as capturing value, and the companies only care about the latter. Making each part of the system more efficient makes the entire system more efficient as a whole, but people are only willing to pay for some parts of the system.
For the browser issue your anger should be directed entirely at LessWrong[1]. Browsers are one of the few types of software which are still hyper-optimized, but there’s only so much you can do when people decide to render a complex application client-side using 6 MB of JavaScript:
Inactive tabs use memory because the alternative is losing your state when you switch tabs and having visible lag and reseting the state when you switch back. It’s possible browser makers are taking the wrong tradeoff and users don’t want tabs to remain in memory, but they’re not being inefficient, they’re optimizing for UX over memory.
Similarly, Electron is highly optimized for what it is; there’s just limits to how efficient you can make a browser while still presenting a fully standards-compliant JavaScript and DOM implementation. Also Electron can’t stop people from writing inefficient JavaScript.
Assuming you think the LessWrong team should stop doing some of the other things they’re doing and optimize performance instead. I disagree but this is a value judgement.
I don’t think they actually could! Even if software engineers really wanted to (which they generally don’t) and had the skills to (which seems to be becoming rarer), the software belongs to corporations, not to the engineers, and the corporations would never let their engineers optimise their software like this. (And if they did, they’d switfly be outcompeted by other corporations that could ship faster more featureful software.)
I think “ability to write efficient, optimised consumer software” is essentially no longer accessible to our civilisation for inescapable economic reasons, just like “ability to build beautiful architecture instead of featureless undecorated glass rectangles”, “ability to build interesting cars instead of bland blob-shaped automatic five-door huge-touchscreen front-wheel-drive SUVs”, etc. etc.)
I agree this tradeoff is definitely a factor—but I don’t think it’s the only tradeoff. We’re also trading-off things the average user doesn’t understand or notice (efficiency, privacy, reliability) for things they do notice (features, a fast release schedule, fancy graphics/UI, network-effects). That’s why the Microsoft Windows “start menu” is now a React app.
I think this results in worse software but that it’s inevitable and out of our control, not a choice where we could choose to do it differently if we wanted to: the corporations that control almost-all closed-source software would never let us optimise their software for things their users barely even noticed, even if it made the software better for those same users, and if by some miracle a corporation did let us they’d rapidly lose all their users to rival corps.
The resources are only abundant so long as we keep pounding away on the upgrade treadmill and never fall behind; today’s high-end phone or computer is tomorrow’s useless e-waste, and that’s a problem that should be (in theory!) entirely fixable in software. The reason a 5-year-old phone takes ten seconds to open a web browser, or a 5-year-old-video card can’t play a modern AAA computer game, or a 7 or 8 year old PC can’t even fit Microsoft Windows + Google Chrome into RAM is problems with the software, not with the hardware.
(Obligatory Linux/FLOSS mention: there does exist super-optimised software that has all the superficial features users notice and all the under-the-bonnet features that make the software actually good/effective to use and it’ll run snappily on a fifteen-year-old computer and it’s free. Why consumers don’t seem to want to touch it is a mystery to me!)
I’m not really sure if we disagree on the high level situation.
You mention that corporations won’t pay their engineers to optimize software, and that a corporation that does would be outcompeted because users prefer software with more features over optimized software, and you even note that in some cases better optimized software with fewer features is free and users still don’t choose it.
I think you’re overestimating how hard it is to write optimized native apps though. It’s tedious to write 5 native apps instead of one Electron app, and the frameworks are worse and slower to work with, but it’s not that hard (also if you need to display anything, you might need to embed a browser anyway). And I think AAA games are not a case of software being less optimized. I expect that on the low end, game developers are putting less effort into optimization, but AAA games on high settings don’t work on old video cards because new video cards are magic (real time ray-tracing!).