Legacy Systems and Distributed Monoliths

We can't have all the software in one system, even if we try. By nature, distribution will always happen. Decades ago, SOA set the standard for creating proper services that prime interoperability. Distribution is not bad; it's how systems work at scale. When companies buy products, they add more distribution because there is no system in a complete vacuum. Now, when we have distribution instead of modules in a monolith, we need to have a way for the system to communicate. When we have a proprietary system, communication options can often be pretty limited. Ideally, communication is thought ahead of time in an approach we call contract first. Where we define contracts and explicitly manage contracts aiming for interoperability. Usually done via a common interoperable interface like HTTP or, nowadays HTTP/2 or even HTTP/3. Legacy systems usually are not API friendly and, therefore, limit your ways of comunication. Such limitations can be drastic to the point where you might only have two options or one option, depending on the case. However, one option(which is far from ideal) is to integrate via UI; yes, people do that, which is far from ideal. The other option is integrating via database; again, it could be a better outcome. You could be wondering, what's wrong with data integration? Companies do that all the time, right? There is a big difference between the big data approach, where data is being ingested, and contact integration of a real-time online system. 

Data integration

Data integration is very fragile. It can easily break. Usually, databases are used for storage, and business rules are in a different layer(usually the application or service). The result is that some business rules might be bypassed. Any schema change that is not backward compatible in any database can break the data integration layer. 
Data Integration


Imagine we have an e-commerce proprietary legacy storefront application written in VB 6, and the database is ORACLE. However, payments are processed with some other proprietary solution, which is written in Delphi, and the Database is DB2. Now, there is a web interface using Pearl and MySQL and a database. For the web system, we have the source code, but such applications still need to talk with the storefront and the payments legacy systems. When the opportunity came, the team decided to write from scratch using Flutter and Postgres as databases, they were good choices, but they still needed to talk with the legacy systems. 

Data integration works well if the consumers are just reading data. Once you have consumers who are reading and writing, things get much more complex. Such an approach should never be the first option. But often, when we need to deal with legacy systems or bad products that are not well-architected, we might need to go over the data route. We have a distributed monolith here, full of fragility and painful maintenance.

ESB / BPEL

People easily think that such a problem needs to be "orchestrated," but I would say that, on the surface, it's not far off. So easily, people think they can replace the data integration with an ESB or BPEL(sometimes both together) in something like this.
ESB / BPEL - A Very Bad Idea

At first glance, it looks nice and might feel right, but it is not. Sure, you centralized all integration on that box, but really, you need to ask yourself if it is worth it. But what you need to keep in mind is that such solutions are terrible. They were bad 10+ years ago; they are bad now. Unfortunately, bad ideas die hard. That's why Jim's Webber Guerilla SOA is still relevant.

ESB/Bpel systems are bad because:
  • They don't scale
  • Increase the Complexity
  • Their assumptions are wrong:
    • XML/Json is better than coding
    • A Business Analyst can just "orchestrate flows"
  • Such tools completely forget error handling and observability
  • Single Point of Failure (SPOF)
  • Bottleneck for Change
  • Does not fix the problem of the root cause
We love tools, and companies love buying software, but (and I've been saying this for decades) buying tools is not how we fix problems. No matter what vendors tell you. Let's also don't forget that:
  • You can't buy integration (sure, there are connectors), but your systems are often unique with unique business rules.
  • We can see ESB and Sync layers, which SYNC will never end. 
  • Handling ownership is always messy; there is an error; where is it? Who is responsible?
  • This can easily result in a bad user experience (more layers, more departments, less isolation, more noise, slower things move, and harder it is to perfect it).
Ask yourself this question: If you need to code all the business/integration logic, what is the ESB giving you exactly? Do you really need it? 

Code Orchestration

We can make this a little bit worse. Consider the following architecture:

Code Orchestration - A bit better


By simply having services to orchestrate, we can improve it, save money, and reduce complexity. We don't need any special technology; it can be done with a simple backend language like Java. We just need a simple HTTP server and proper contract definitions. ESBs are usually given as a "free" gift, like any Trojan horse. Vendors have applied such a strategy for decades. They still do that.

Now the question you need to ask yourself:
  • What's easier to handle an error—having the code and understanding what happens or not having the code and having a bunch of XML/Json/Yaml to debug? 
  • What's easier to code? Coding with a programming language or a sudo programming language where you drag and drop geometric shapes? 
  • How easily can you instrument? When you have the code in a simple service or when you have a proprietary solution that doesn't let you instrument anything? 
  • When something goes wrong, what will be easier to debug? Your service code or some ESB/BPEL with code in XML/Json/Yaml?
Not all products are good products. We always need to think beyond the happy path. That's why it is always good to do a POC and simulate chaos and failure scenarios just to see how easy it is to debug, troubleshoot, instrument/observe, and even test it. Can you test it? 

Integration vs Service Orientation

Is integration always a good thing?

IMHO, No. We should only use integration as a last resort. What we should have is interoperability. Good systems should be able to talk to each other without the need for a third system. If we need three systems to make two systems talk to each other, we have a problem. A big thing in SOA is service orientation, which means services, not integrations. Services need to be able to talk to each other alone, without any intermediary. 

The more integration you have, the more fragility, the more points of failure, and the harder things are. We should be considering fixing the problems at the root, which is enabling interoperability and having better systems that can talk to each other naturally. IF the systems cannot communicate because their architecture is not good, and that is the root of the problem.  But there is more...

Fixing the Wrong problem


The reality is that we buy tools that don't fix the root cause of problems. We entertain ourselves with technology and another level of indirection. More complexity and more bad systems do not fix the problems. The real problem is old legacy systems(which are distributed monoliths) and technical debt that is not being addressed, and that's what should be addressed. Instead, we pile up bad systems on top of bad systems. This problem is not new; not the first time I have blogged about this in 17+ years. Pretty sure it will still exist in the future. Waiting for AI-powered ESBs :-) 

There is no magical answer. We need to pay technical debt and write better systems. How we work and how we build software matter. We can do better by shifting some mindsets and doing:
  • Avoid buying tools that do not fix the root cause of your problems.
  • Better evaluating products, via tradeoffs, considering complex pocs with error handling, chaos, and troubleshooting.
  • Writing simple services rather than buying ESBs
  • Thinking about observability when we build systems.
  • Have interoperability in mind when we build software. 

This post could be easily written in 2004.

Cheers,
Diego Pacheco

Popular posts from this blog

C Unit Testing with Check

Image
When I start on IT, several years ago(+15), C was the first language that I learned. But was not the first language I mastered and deployed in production which was...surprisingly Visual Basic 6.0. However, when I learned C or any language back on that old and arcane time noddy was into testing. However now is impossible to work professionally in any language without tests. C is a low-level language but still kicks ass and has several interesting libs/frameworks. People use to fear C a lot because of lack of syntactical sugar but once you get used to it is not that hard at all. Today I want to show how you can do Unit Testing with C. I will not show how to do Unit Testing with C++ but with C ANSI actually. We will be using Check a unit test lib for C ANSI.
read more »

Having fun with Zig Language

Image
Zig is a general-purpose language created in 2016 by Andrew Kelly .  Zig aims to fix the issues C language has while being pragmatic and simple at the same time. Zig really shines as being simple by not having any hidden control flows, not having hidden memory allocations, no pre-processors, and no macros. Basically, it's one language and one language only. Since pre-processors and macros can get really complex and become their own universe. Sure it's not the Zig Philosophy.  Zig Compiler(Zig cc) also is a C compiler, and the interoperability from C to Zig is very good. Clearly, we can see influence from other languages like Go and Rust; however, Zig is pretty unique and different from Go and Rust.  Zig is fast. Zig is perfect for system programming. I bet we will see a lot of solutions being built in Zig. As a languages aficionado, I like to learn at least one language per year and sometimes more; Zig was my 2021 pick, did not regret it. Even if you dont use all language...
Keep reading

HMAC in Java

Image
In 2018 AWS CTO(Vogels) said : "Security is everyone's job". For the last 2 years, there was so many famous and big data leaks and breaks that made that statement be very true more than ever. Security often could mean worst performance and worst user experience so in order to get it right you really need to think about the designs before jumping into to code and consider performance and user experience has main requirements.  I was thinking about writing about how to do HMAC in Java for a while and recently Redis 6.0.0 come out and to my surprise, there was a refactoring on the password part in order to use HMAC.  Security easily could scare engineers often because it is not something well spread yet but I believe this will change soon. It does not matter if you have to deal with PII Data or not, security is super relevant because everybody is running their workloads at the cloud or with IoT and Edge devices which means more distribution, more code, more points of fail...
read more »