AI Coding Agents Economics
Let me be honest: I don’t think that using AI you can get 3-5x the productivity from engineers right now. There are so many things I need to debunk here; it’s even hard to choose one to start with. But for a moment, just. For the sake of argument, let’s believe it's true(just for a moment). That will help you to understand why AI has suddenly become so appealing to companies.
In the past, software was always more expensive than people, so people tended to optimize for software, resulting in efficient, optimized, and robust software. Those times are behind us, because at least for now, people are more expensive than software. Tokens continue to get more expensive, and cloud computing is also very expensive, yet people are still more expensive.
Why am I saying this? Because if people are so expensive and due to AI, you 1 engineer using AI can do the work of 3. Now you can do amazing savings. I’m not saying I agree with these numbers or even that I’m in favor of fewer good engineers; I'm just explaining the “thinking”.
The fallacy: More is More
Keep in mind the thinking I just explained. Now You think. IF the company has money, actually, it’s better to hire more engineers. IF one engineer does the work of 3, two engineers will do the work of 6. Things will go much faster. Now it’s much better to hire more engineers, because if your competitors are hiring, then you will be behind because they produce much more than you.
Such thinking is common and has many, many, many flaws. First of all, you won't get 3-5x productivity. Measuring productivity in engineering is very hard. If you can’t measure, how can you tell you got more? The best metric is LED time, from idea to production. IF that is improving, great, you're doing much better. But delivering faster —even into production —has limitations.
The hidden assumption here is that “engineering” is the bottleneck; if you get more productivity from engineers, then you speed up the whole company. Which might be true in some cases, but I will argue that, in many cases —and even the majority of cases —it is not true.
Dangerous Assumption: More software delivery → Immediate profits
The hidden assumption here is that. Engineers producing more means more profits. Well, engineers producing more means nothing if they dont go into production. Often, companies use release trains(which are an anti-pattern) and deploy in production from 2-4 weeks. Suppose the engineers finish faster due to AI coding agents. Now their work is sitting in a queue, waiting for DevOps/Release to deploy to prod.
Let’s say you have released CI/CD and can deploy to production immediately. Does it mean that if you deploy to production, you make money immediately? In a partnership or deal, maybe, but in product development, you are learning and discovering, and often being in prod does not mean immediate profits. It’s possible and common that you need to deploy many times in production to learn; even so, is possible to deploy all the time and still not learn and still not make profits. Every single project in the universe can fail; it’s no different with AI. Your consumers don’t care if you use AI or not; they care about the value you add to them if your product is good or not.
Dangerous Assumption: You only need Beef-Up Engineers → Local Optimization
Ok, for the sake of argument, let’s say we them really want to beef up all engineers, because they being more productive will have a positive effect on the whole org. Again, if you are not doing it, your competitors will be doing it, and you will be behind.
Let’s say you have real CI/CD and can deploy to production immediately. Can you release it in production immediately? Deploy is often easier than release; release often requires other departments, like marketing or even legal (depending on the industry you are in). Marketing, Legal, and supporting departments are often much smaller than engineering, like 100-to-1, maybe? Let’s say yes, they can be at the speed of light and keep up with engineers using AI. You release fast now. (big if here).
Remeber, other engineers also need to review your code before going to production. Engineering usually does not play alone. IF the other parts of your pipeline, like DevOps, QA, and Marketing, often happen after engineering. QA usually has a 10-to-1 ratio, as does DevOps, which can easily reach 100-to-1. If you have perfect automation, you are gold. What if you don’t have good automation? if not, can they keep up with a super beefed-up engineering team?
Very likely, what will happen is that engineering will flood the engineering review queue(code review will take longer), and then flood QA, and then flood DevOps, and then flood other departments. Because we need to see the WHOLE. IF we are just optimizing engineering, we are doing a Lean anti-pattern called “Local Optimization”; Lean believes that we should always optimize the whole.
Dangerous Assumption: You only need Features
There is another hidden assumption here. All you engineers' time will be used on the feature to achieve 3-5x productivity gains, but that is not 100% true. They need to deal with legacy systems, troubleshoot problems, vulnerabilities, and address environmental issues. Sure, AI can make all that fast, but my point is dont think you will use AI just for features, really not the case.
Let’s pretend AI could just do features. Does having features mean you will increase profits? Thats perhaps the number one mistake of modern feature factories, that features always lead to profits. Features also lead to feature bloat, complexity, technical debt, and slowness. Just creating features slows down feature creation (a common theme in the philosophy of software design).
Dangerous Assumption: We forget the cost of Tech Debt
Companies have technical debt. Some more than others, but all companies have technical debt. Technical debt has several consequences. One of them is that drag engineers' productivity is down. Often legacy systems are full of anti-patterns and have few tests. Again, AI will need to spend time generating tests and refactoring code, which is not easy; refactoring with AI is far from perfect.
Now, again to my point, AI will need to deal with technical debt; otherwise, there is the risk that it will introduce technical debt by copying bad examples and anti-patterns in the code. IF a Junior engineer or a bad engineer is using AI, atually will actually introduce bugs faster and more bugs. So again, you are ingesting poison and faster. All this is affecting your time to deliver features, even with AI.
Dangerous Assumption: Engineers are always READY
Perhaps the most dangerous assumption. Very often, engineers start working on stories without discovery. Where the requirements are ambiguous, or where the product — or even the business — is not 100% sure what it wants.
IF Engineers are waiting for a product with clear requirements, AI is IDLE, not coding features. Think about that. How many product people do you have? How fast can they feed engineers? Usually, the reality is a common tension and back and forth between product and engineering, where discovery and delivery blend, and going into production and running experiments is how you learn. How fast can you learn? How fast can you turn that learning into profits?
Again, if you don’t beef up the product and even the business, can they keep up with engineers using AI?
The reality is that engineers often get items that are half-discovered at best, but many, many times, it’s just a decision someone makes, does not have consideration, does not think about corner cases, does not think about consequences, or if it is even feasible in current systems and current technology. The side effect is that engineers take longer because they need to figureout these things.
Tickets arrive in engineering often NEVER READY, such a reality will be the same for AI.
Dangerous Assumption: We understand the principles of what we are doing
Dr Deming often said that companies copying Toyota would fail because they dont understand the principles behind what they are doing; they end up copying what they say and never get the same results. Now we use AI to speed up engineering, but do we understand the principles of scalability? Do we understand the little law? Are we going faster or just queuing up faster? Do we know how to learn? Do we know if we are actually learning?
Do we know if users use our solutions or even click on our pages? IF we don’t know any of these things, can we know we are making the best decisions that will lead to value added to users and therefore profits for the company?
We need operating principles, we need operating principles with AI. Without such operating principles, we would very likely be doing it wrong, as happened with the Agile Movement, the DevOps Movement, and, in the future, AI.
Two Steps Forward, One Step Backward
We also need to remember that AI hallucinates, that AI has downtime like any software(because it’s an API). AI can have bugs, and those bugs can degrade the model's quality. Combine it with the points I mentioned earlier, and you'll have many things dragging you down, so you won't go as fast as you could.
So, should we ignore AI? No. I do believe we can achieve 10-30% gains in productivity from AI and improve engineering. Now, the big lesson here is that we need Lean thinking. We need to see the whole, we need to use value stream mapping and beef up the whole company, then we can all go faster. Don’t believe me? Read:
- Google CTO said engineers are 10% more productive with AI
- Goldman says AI’s influence on productivity where it’s already been deployed is pronounced. Hatzius’ team cited academic studies and company anecdotes indicating generative AI adoption delivers, on average, a 23%–29% boost.
- AWS CEO Says Replacing Junior Developers with AI Is the Dumbest Thing He's Ever Heard
The second thing here is that we need to remember that software is about learning, and product development is about learning. To increase profits, we need to learn more about our customers —what they want, how they behave, what pains they have —and how we can create the best software to add value to them. We also need to think about how to beef up learning. How do we learn faster? That is much more than just AI typing faster on the terminal on behalf of some engineers.
Still think AI has great potential, but only if we learn from the mistakes of the past. Before AI, Agile would transform companies and they would produce more; DevOps would transform companies and they would produce more; Cloud would transform companies (yeah, downtime still happens); Digital Transformations would fix companies. Will AI fix companies? Well, it’s all these movements wrong, or maybe the problem is that we have not learn how to learn yet?
cheers,
Diego Pacheco
