Reflections on Hard Decisions: Avoiding Coupling, Denial and Dogma

Previously, I was blogging about two very HOT Button Issues (The Death of Microservices and Why Waterfall can be better than Agile(Sometimes). Today I want to expand and correlate these two topics a bit more. It's important to say again that I like agile, and I do it for +14 years. I have no problems talking about Agile issues. You shouldn't either. Hard questions sharpen our solutions. There is a huge difference between go trough a set of hard questions to justify investment harder than power or politics game to archive a result. The difference is one is healthy the other is not. There are "Healthy" matters like: Coupling, De-coupling, execution, Agile / Waterfall we should be able to navigate, talk, analyze trade offs without dogma or any issue whatsoever. However is hard to us to talk about some subjects right? What happens is that there is so much FEAR and TABOO on companies that some subjects cannot be ever on the table. Effective Learning means, we can talk about things, effective leaders allow it, they don't hide it. The only way to do proper due diligence and take hard choices is with taboo-free environment. The market / IT-Industry is not quite there yet. Getting back to the Agile. Why are we talking about not doing agile in 2020? Sound wrong right? 

Did you realize Agile has issues? 

In my previous posts, I was not taking hard on agile. However, I was talking trade offs with several explanations, and it bothers people. Why? We are in 2020, and we still cannot understand that Agile is not a Religion? Therefore, Agile can and has defects! In not saying is right or wrong, but some people do not believe in Good / Religion, and I get it, there are idiots everywhere doing the wrong things in all possible fields. Easily we can say that Agile is the ultimate Religion because it has much less criticize. 

Waterfall Execution VS Waterfall Contract

95-99% of all times, we want, and we should do Agile. However, there are 1-5% of cases where agile is worst. However, this is too abstract, and we need to make a difference between a couple of things. 

#1 Talking about Contracts: Waterfall contracts meaning (Fixed scope, price, and time) are bad for all parts we should never do, and I sincerely believe they are evil—bad Deal.

#2 There is a difference between Waterfall execution and waterfall contract. Ideally, you want to do incremental execution because there are so many benefits:

• Risk Reduction
• Ability to Change direction
• Reduce Waste

However, some problems do not have this Incremental nature or incremental benefits. There are some examples like Real State, Building a Boat, or making a Video game, for instance. A video game is a pretty waterfall because you need to say when the game will be released, and it needs to be lunch on that date, which got released one year ago or more. 

For a boat, there are no much increments you can do as well, either you have the whole ship or you don't, there is no incremental into production. I'm not saying we should not do Agile, We should, as much as we can, however, some problems are coupled by nature like Buildings, Boats or Games, and there is nothing much else we can do. 

Could we mitigate risks with Waterfall? YES. By doing:

• Discovery and UX Validation Work(Before we deliver)
• Good and Extensive Design work
• Planning and Estimates

You might be wonder, that are things agile fought for decades(Waterfall execution, BDUF, Analysis Paralysis, Documentation, and even Planning and Estimates and they are correct; most of the cases, you should not work this way. However, there are always different scenarios. 

Right, but what are the issues in Agile?

• Lack of Discovery/product focus and practices
• Agile was made for Teams, hard to scale to company level.
• Lack of Architecture (Scale and you will see the PAIN in your Spotify model)
• Poor DevOps and SRE practices and focus.
• Coordination is an issue - The same problem in microservices is push to blank spaces or upstream(like in your Spotify model). 

Don't get me wrong agile manifesto was around 2K year, so there was no:

• DevOps Movement
• No Really Digital Products
• No Cloud
• There was Architecture, but they ignored - Big Mistake

That's why MODERN AGILE was born pretty much. But again, some issues in "original agile" does not reflect all the things we need in 2020. Also, that's why in 2012, there was a rise in DTA(Dual Track Agile) and others multi-track agile forms because there are gaps. 

Different Reasons - Same Coupling

(A) Are you a Good Leader / Executive? If you SELL by delivering software on exact dates one year before you "announced" on an event, are you the right Sales Person, or is your Tech releasing on time doing the selling? What drives the adoption of your choices, our guessing skills? Do we need salespeople for Games? In software word sales is on death spiral for decades, it's not sales who sell, easily is Tech itself or marketing but not sales. 

(B) Do we have real coupling or induced coupling by years of technical debt not being paid? Sometimes the problem is a couple by it-self, but I would argue thats not the majority of cases. Tech debt is like poison, slowly kills your ability to:

1. Be productivity 
2. Reason about the Code
3. Deliver new Capabilities
4. Retain People
5. Re-shape your product/focus

(C) Natural Coupling - may be on a Building, GAME, or an in a Boat. However, there are some cases where both can be Incremental, therefore agile, and have value. For a boat, you might say let me give you the BOAT, and you decorate as you want, them we DE-COUPLE the problem in 2 boats as infrastructure and ship as decoration/customization. The first part will be a Waterfall, but the second can be more agile. The same could happen with a building. For TellTale (which bankrupt btw), you got games released by chapters every X weeks or Y months. They did this for TWD and Batman(Pretty good games BTW). However, you CANT do this for all games. You might argue DLC(Downloadable Contents) are ways to make the games more agile, accurate. However, the CORE of the game itself will be a pretty waterfall. My point is, increments are not always possible; you should be Agile and look for increments when they are real and potential. 

So, Should we always do a Waterfall? 

No. Agile should be the Default. But we should have no Dogmas about it. We should be able to see the weakness and do trade offs analysis all the time. IMHO Agile can work 95-99% of cases. Do I hate agile? No. I love it. I think it is a good thing and you should use it, don't be blind about it. 

Easily someone could argue, Buildings(Apartments, houses, bridges) and boats are not software, they are hardware. Hardware projects(In software - IoT, Infrastructure, Appliances) have similar properties; even when you have a software part, your execution will be Waterfall or part-agile only if you depend on hardware. There are gates, you can, and you should be as much agile as possible, but it's unlikely to have a company 100% agile. It's okay. Why we always need to be ALL THE THINGS, ALL THE TIME? 

Hard Decisions

My main issue with microservices is that people did not understand the principles and always want to eat the cake and have the cake. A similar problem happens with Agile and with discussions like: Should we Re-write or should we refactor. There is no One Size Fits all; there is no silver bullet; Decisions need to be tactical, case-by-case bases. 

There will be scenarios where re-write is BS(Netscape case); there will be scenarios where re-write is the sensible thing to do(Ramesh case). The hard decision is not to make easy and comfortable choices for you, but think trought options and trade offs and have a balanced and sensible choice. 

IMHO the right decision is also balanced between Business, Product, and Technology. If technology or business pays the price all the time is a wrong balance for sure. 

Cheers,

Diego Pacheco