Proper SOA Isolation: The 3 frontiers
The Lack of Isolation
The lack of isolation can be seen in many forms of technical debt and bad traits such as:
- Fragility - Changing 1 thing in the service is very difficult and takes a long time, and it's very easy to break yourself and consumers all the time.
- All or Nothing - It's difficult to introduce local and incremental changes. Often required to migrate a bunch of other services/libs or your whole ecosystem.
- Never Up to Date: Always struggling with technical debt, migrations, legacy software, 2-3 majors or more versions behind.
- Do nothing or Make it worst: You might not be able to fix the problems by your team because it impacts several other teams, or you require multiple changes from others, or by fixing it, you create a new service but use the same database. Therefore, you are making things worse, not better.
- Release Trains: Release trains should be temporary; why you cant easily apply CI/CD. It's not a tooling problem; it's a lack of isolation problem. Why do services need to be in SYNC? Well, because maybe they are not isolated enough.
Independence it's a highly desirable property in distributed systems. However, you cant achieve independence without having isolation. Isolation is a pre-requirement for independence.
3 Frontiers of Isolation
In march 2022, I tweeted about SOA Isolation and 3 frontiers. Today I want to cover this subject in more depth and explain why it's critical and how we can do better in regard to Services.
https://twitter.com/diego_pacheco/status/1504000827974770691
In order to have isolation, we need to have isolation in the 3 frontiers of a service: Contracts, Databases, and Internal Shared Libraries. You can only have isolation in service if you have these 3 elements isolated. Databases are getting obvious for many in our industry, but just databases are not enough. It's easy to get Distributed Monoliths when Isolation needs to be applied properly in all these 3 frontiers.
What is Distributed Monoliths?
I can say that when companies went all-in microservices without following proper isolation principles, the result was distributed monoliths. Companies were trying to get away from the monoliths, which often were and still are:
- Full of Technical Debt
- Difficult to Change
- Lacking Testing and Coverage
- Using Old/Legacy technology, in other words, dated
- Not pleasant for engineers to work on - affecting dev morale and turnover
- Difficult to operate - all or nothing
- Lacking resiliency and sometimes even stability
Microservices were the way to fix this problem. But people should have realized that Microservices are more complex and require more discipline.IF people needed to have discipline on the monolith, why would they have it with more software, more distribution, and less scrutiny? You know the answer.
I dont blame microservices. I believe they are a powerful style of Architecture and make sense. However, they are not all or nothing. It does not matter the granularity you pick if it is a FaaS/Serverless, Microservice, or big SOA Service - without isolation, there will be trouble. A distributed monolith has all drawbacks classical monoliths and microservices have. Distributed Monolith is an Anti-Pattern. Unfortunately, they are pretty common in our industry.
How a Distributed Monolith looks like?
Distributed Monoliths are tricky because they will give you some sense of independence but not really 100%; At the end of the day, if you can't perform local changes in isolation and have full deploy independence, you have a distributed monolith - even if:
- You have Different Code Bases - Different repositories in git
- You have Jenkins pipelines per services
- You have different teams per services
Because code isolation is not enough. We need to isolate the 3 frontiers being: Contracts, Databases, and Internal Shared Libraries.
Isolating Databases
This is the most common mistake companies make. They create new services, but all share the same databases. Fixing this problem is hard. But possibly, there are patterns and approaches we do to fix database sharing. Like the Strangler Fig Pattern and many data migrations patterns.
Having 2 services accessing the same database is good but if they are from different domains is even better. IF services are from the same domain, it might mean the same people and same directors and VPS and, therefore, easier to perform changes in isolation. Lack of database isolation is base because it creates high coupling and forces services to be in Sync; otherwise, they can break themselves very easily. Does not matter if database access is performed in a direct access form or via an internal common shared persistence library - the issue is the same.
Contracts: matter more than implementation
Services are composed of 2 elements. The Service Contract and the Service implementation. Contracts are the most important piece. Period.
Isolating Contracts
People tend to ignore contracts and pay attention to implementation. Yes, engineers spend most of their time on implementations, but contracts are far more important. A proper contract should be naked. Meaning no dependencies, no libraries, no frameworks, or as few as possible because this is another form of binary coupling.
Drivers/Clients can be dangerous as well. They sometimes blend with contracts. The same rules apply; they need to be with few dependencies as possible. Copy and paste code is far less evil than having binary coupling; people always think about the case that a bug will be happening over and over and therefore re-use is a must, but they forget the acknowledge that overtime reuse will charge a high price during migrations and big feature implementations.
Isolating Internal Shared Libraries
This is the most controversial yet big problem all companies face, but only some realize how bad it is. Internal common shared libraries, most of the time, are evil and bring more problems them solutions. I lost count of how many companies, products, and projects I worked on that had bad libraries. They are everywhere in the industry.
The solution is not to stop building libraries but do it so with much more du-diligence. When you build a service, you have a contract, and if that contract is stable, naked, and properly isolated, you have a lot of flexibility to change things under the hood. Changing the whole service language and databases is possible without breaking consumers.
Common Internal Shared libraries need to be Lean, cherry-pick dependencies very well, and be manageable. Therefore I would argue you do not want to have hundreds to thousands of libraries. Otherwise, you can't manage them and will always be behind.
The Way Forward
The way forward is to have independence based on Isolation. Isolating Contracts, Databases, and Internal Shared Libraries. This can be achieved by:
- Constant training and continuous education of engineers
- Constant Architecture and Design Reviews - based on principles
- Prioritization of Principles by management instead of unlimited freedom
- Scrutiny on Internal Shared Libraries, especially if affects the whole org
Services should not be taken for granted. Just because you are exposing some data via a REST interface, do not assume you got the principles right, and you are done. Principles need to be encouraged at all times, and if you dont have enough talent density, thats fine, but then double down on education. I would double down on education no matter what.
Cheers,
Diego Pacheco
