Proprietary Systems and Distributed Monoliths

Distributed Monoliths are the predominant form of modern legacy systems. Sometimes distributed monoliths are created by teams misapplying microservices architecture due to lack of isolation or internally shared libraries break of isolation. Other times, there is just the proverbial no time to refactor. Let's share the database. However, sometimes distributed monoliths happen due to non-obvious reasons from decisions that, at first glance, sound like good ideas and are even far away from engineering. 

In software, we often run a trade-off analysis, Build vs Buy, where we must decide whether to build or buy software. Buying is usually attractive because it gives some speed (since you don't need to code it). Building, of course, takes longer, and you will need to maintain it; there is a chance of the project failing, and you invest money and have no software. There are trade-offs between building and buying. However, people often think buying has no implications and is much better.

Buying Software

In reality, at scale, things are more complex. When companies buy software, they create a lot of non-obvious problems, which usually do not show up immediately; what shows up immediately is:

• Perception of Savings: We paid much less to vendor X than they would if we built it.
• Perception of Productivity: Now we have this new software from vendor X. We have all these features we did not have to build, and we can focus on something else.
• Perception of Less mantainance: Proprietary, closed-source the software you buy, you can't change the code. Therefore, it is not your responsibility to keep up with vulnerabilities; consequently, you might think it is easier to maintain.

Good software products can definitely deliver all these items. However, not all products are created equally. You can't change the code when you buy proprietary, closed-source software because you don't have the code. It might sound obvious, but no software is an island! Software must be aggregated, augmented, and integrated with other systems to drive value. That's when the proprietary closed-source software can be very challenging if no APIs are available because you still need to integrate, so what do you do? 

Throwing Money

The most common solution(which is not always possible) is to throw money. Pay the vendor to build your feature and address your needs. The problem depends on your internal system; such a move could be costly and take a long time.

Web Hack (IF you are lucky)

Great products have decent APIs. But it's possible that some proprietary closed-source systems do not have APIs, and there is no way to throw money; not all vendors want to do features. If you are lucky and the product has a web interface, you could use some tool like Selenium, Playwright, or Puppeteer to automate something via UI. The problem is that your scripts can break when you get an update, so this option is awful, slow, and painful.

People do that, even for desktop systems. Although it is often used for testing, it could also be used as a form of integration.

The Database 

Often, the solution is to pass the proprietary closed system and call the database directly. That is when we create a distributed monolith. The more proprietary closed-source systems you have that don't have APIs, the more database hacks you need to do, and the more distributed monoliths you will have.

Some people call these "Satellites" but do not fool yourself. This is a distributed monolith.

Proprietary closed-source systems often are tricky to debug and troubleshoot as well. Because you are boxed with limited access and tools. 

Doing Better

To do better, we need to understand that buying software is not always better; it creates problems, and such issues can be mitigated by:

• Think about the cost of integration on trade-off analysis for build vs buy.
• IF you can throw money, ask the vendor to build APIs.
• Buy modern solutions that have decent APIs. 
• A lot of modern products are open-source, so I favor that one.
• Sometimes, buying the code is possible; it's a good idea because you can change it and expose proper services.
• Understand and learn key principles of distributed systems to avoid creating distributed monoliths. 

Principles of Software Architecture Modernization is full of examples, principles, and techniques on how to deal with monoliths and distributed monoliths at scale. Continuous Modernization covers the mindset, practices, and shift to a better work day with teams dealing with such systems.

Cheers,

Diego Pacheco