I enjoyed this.. fun sort of intentional-meandering style.
I propose that scaling (as defined here) is a general problem, not just a software problem. I realize this is a hard case to make. Technical debt, refactoring hells and such don't plague film sets or factories like they do code shops. But.... Consider this:
- Consider factories. A factory is just a huge pile of physical capital organised to efficiently do a specific, defined task. Everything is optimised to reduce marginal costs. Capital efficiency is strictly enforced by capital money markets. Marginal cost efficiency is strictly enforced by real markets.
... Either of these two constraints (marginal costs too high or not enough capital) usually come into play before engineers have had a chance to redesign, refactor, repurposed and abstract a factory to the point where these are a problem.
- Everything that is in a movie is in a shot. Movies have a finite number of shots and a director can direct each one. A film may have a whole team replacing Toby Philpott with Jabba the Hutt, an army of costume designers, set designers, actor psychologists, etc. But, everything that gets into the movie gets there in a shot, and the director can work shot-by-shot and wield god-like control of lots of labour that way.
Software is unique. Software is free from most physical limitations. There are no material costs. There are no capital costs. The only economic resource that a software enterprise wields are the engineers/engineering itself.
When a Ford Company scales, it raise money. It uses that money to to build factories, buy materials, etc. These are all scarce/finite resources needed to start/continue making additional cars.
Google, MSFT, FB, post-AWS amazon... When they scale up, they just scale up. They hire more engineers. They produce more software. The only "resource" being scaled up is the people making the software.... Something has to be the limiting factor.
In any case, the "scaling creative work" problem does exist in factories too. The difficulty 50s era auto manufacturers faced competing with Toyota are sort of evidence. They struggled with "technical debt" in the form of car models, factory design, company culture and such that couldn't adapt flexibly.
A lot of tesla's wins (besides marketing, fundraising and software) have come from their recent blank state start. Auto manufacturing today is highly caught up in the "Toyota Way" of doing things. It's been that way for decades and parts of it are explicit assumptions of international trade deals. It's very compartmentalized. Flexible within compartments. Rigid without compartments, whether they are outside the department or outside the company. It's a lot like the city metaphor this article mentions.
The problem is... sometimes you need to design the factory and the car simultaneously. At times that has screwed Tesla. The quality control benefits of the Toyota Way are not to be taken lightly. At other times though, it works. The factory designer has been banned from designing the car and the inverse for 40 years. I think their version of technical debt is the reason.
I propose that scaling (as defined here) is a general problem, not just a software problem. I realize this is a hard case to make. Technical debt, refactoring hells and such don't plague film sets or factories like they do code shops. But.... Consider this:
- Consider factories. A factory is just a huge pile of physical capital organised to efficiently do a specific, defined task. Everything is optimised to reduce marginal costs. Capital efficiency is strictly enforced by capital money markets. Marginal cost efficiency is strictly enforced by real markets.
... Either of these two constraints (marginal costs too high or not enough capital) usually come into play before engineers have had a chance to redesign, refactor, repurposed and abstract a factory to the point where these are a problem.
- Everything that is in a movie is in a shot. Movies have a finite number of shots and a director can direct each one. A film may have a whole team replacing Toby Philpott with Jabba the Hutt, an army of costume designers, set designers, actor psychologists, etc. But, everything that gets into the movie gets there in a shot, and the director can work shot-by-shot and wield god-like control of lots of labour that way.
Software is unique. Software is free from most physical limitations. There are no material costs. There are no capital costs. The only economic resource that a software enterprise wields are the engineers/engineering itself.
When a Ford Company scales, it raise money. It uses that money to to build factories, buy materials, etc. These are all scarce/finite resources needed to start/continue making additional cars.
Google, MSFT, FB, post-AWS amazon... When they scale up, they just scale up. They hire more engineers. They produce more software. The only "resource" being scaled up is the people making the software.... Something has to be the limiting factor.
In any case, the "scaling creative work" problem does exist in factories too. The difficulty 50s era auto manufacturers faced competing with Toyota are sort of evidence. They struggled with "technical debt" in the form of car models, factory design, company culture and such that couldn't adapt flexibly.
A lot of tesla's wins (besides marketing, fundraising and software) have come from their recent blank state start. Auto manufacturing today is highly caught up in the "Toyota Way" of doing things. It's been that way for decades and parts of it are explicit assumptions of international trade deals. It's very compartmentalized. Flexible within compartments. Rigid without compartments, whether they are outside the department or outside the company. It's a lot like the city metaphor this article mentions.
The problem is... sometimes you need to design the factory and the car simultaneously. At times that has screwed Tesla. The quality control benefits of the Toyota Way are not to be taken lightly. At other times though, it works. The factory designer has been banned from designing the car and the inverse for 40 years. I think their version of technical debt is the reason.