DevOps Monolith

DevOps Monolith
The term Monolith it's often used in legacy systems or system poorly designed and poorly architected.  Microservices are the facto stand architecture when we talk about software. There are lots of companies also doing DevOps Engineering and DevOps per se its related to microservices.




Microservices required some infrastructure work like for instance:
  • Provisioning: Install OS system packages, files, scripts.
  • Telemetry: Dashboards and alerts for you microservice eco-system.
  • Testing: Stress Testing, Chaos Testing, Load Testing, etc...
  • Canary: Automated canary analysis, deploy, score, and rollback.
Infrastructure work could be more or less depending on your level of abstractions for instance if you are working with bare metal or IaaS you will definitely have more things to get done. However, if you are doing cloud-native microservices using kubernetes or any FaaS stack you might have less work but always well have some infrastructure work for someone to do.

DevOps is a really interesting movement. There are lots of folks talking about cultural aspects like Blameless incident reviews, CAMS, and Lean. This is an important aspect that doesn't get me wrong.  However, there are some engineering aspects around DevOps Engineering that need more care like avoid coupling for instance.


Infrastructure is Software
Lots of infrastructure work are basically API calls. Create a machine is an API call. Define a network(SDN) it's a set of API calls, creating load-balancer is an API call. Lots of infrastructure work today are resume as API calls in other words software. So in order to have software we need the code. That's the easy part. However, Engineering is more than just code. It's also about testing and equally or more important architecture aspects.


There are great tools on the market today to do provisioning like Ansible and Terraform for instance. Both solutions work in a declarative fashion where you describe the state you want in the machine for instance. Both solutions have ways to share common infrastructure code.


Every single engineer what's to avoid duplication because duplication often means more bugs and less productivity. So these tools have ways for you to share this code on multiple machines. So let's say you will do microservices with Netty for instance. You will have to do some OS optimization in the sense of TCP and Open Files for instance. So you want to share that. Ansible allows you to create a role in order to share that with several other infrastructure components. That sharing is not only about configs but installations. So let's say pretty much all machines will need to have the same telemetry configurations or share the Tomcat app server. Why provisioning the same thing all the time right - let's just share it right?


The DevOps Monolith


The DevOps monolith enters the room. It all started as a good idea and then turn engineering life into a nightmare, I see that a couple of times already, so what problems are created when we see reuse as a blind thing.


Yes, the monolith is not only about Application but today there are Infrastructure Monolith or DevOps Monoliths. Infrastructure got great things about code it also gets some of the most painful issues as well.


Before talking about some not so nice things, let's talk about good things, so what are nice wins you get when you share infrastructure code:
  • Standardization: You know what directories to look, where scripts are, and how things work so this is definitely a big win and reduce engineering cognitive load for sure.
  • Productivity: Less work to do at the end of the day. It's faster to investigate things. Required less documentation since things are in the standard place.
DevOps monolith consequences:
  • Build Time: I see scenarios where provisioning could take up to 40 minutes. Yes 40 minutes to build an AMI. This kills productivity.
  • Coupling: Hard to change something just for your microservices without affecting others. Maybe you couple some Os dependency and it will be hard for others to update.
  • Side Effects: That's another experience I had when using the same roles for microservices in things that are not microservices like NoSQL databases. They need different configurations, tuning, and no necessary same libs. So it's hard to predict what's happening and also a source of bugs.

Back to Design and Trade-offs
There is no easy solution to this. My experience showed me using shared software blindly is a bad idea but also rewritten everything does not work as well. DevOps monolith is dangerous because it can kill all main microservices benefits like speed to deliver software and the ability to work with different things(different times to update libs and dependencies).


Share nothing for a small business looks like an overkill to me however share blindly on medium / big company it's also a source of lots of painful issues and source of many Technical Debts.


There are trade-offs that need to be taken into account. This is no different than them issue we have with the software. As we learned that Software architecture is best with isolation where services do not share: OS, Database, and Code. We might need to start introducing this isolation idea on DevOps Engineering as well.  
Cheers,

Diego Pacheco

Popular posts from this blog

C Unit Testing with Check

Image
When I start on IT, several years ago(+15), C was the first language that I learned. But was not the first language I mastered and deployed in production which was...surprisingly Visual Basic 6.0. However, when I learned C or any language back on that old and arcane time noddy was into testing. However now is impossible to work professionally in any language without tests. C is a low-level language but still kicks ass and has several interesting libs/frameworks. People use to fear C a lot because of lack of syntactical sugar but once you get used to it is not that hard at all. Today I want to show how you can do Unit Testing with C. I will not show how to do Unit Testing with C++ but with C ANSI actually. We will be using Check a unit test lib for C ANSI.
read more »

Having fun with Zig Language

Image
Zig is a general-purpose language created in 2016 by Andrew Kelly .  Zig aims to fix the issues C language has while being pragmatic and simple at the same time. Zig really shines as being simple by not having any hidden control flows, not having hidden memory allocations, no pre-processors, and no macros. Basically, it's one language and one language only. Since pre-processors and macros can get really complex and become their own universe. Sure it's not the Zig Philosophy.  Zig Compiler(Zig cc) also is a C compiler, and the interoperability from C to Zig is very good. Clearly, we can see influence from other languages like Go and Rust; however, Zig is pretty unique and different from Go and Rust.  Zig is fast. Zig is perfect for system programming. I bet we will see a lot of solutions being built in Zig. As a languages aficionado, I like to learn at least one language per year and sometimes more; Zig was my 2021 pick, did not regret it. Even if you dont use all languages in p
Keep reading

HMAC in Java

Image
In 2018 AWS CTO(Vogels) said : "Security is everyone's job". For the last 2 years, there was so many famous and big data leaks and breaks that made that statement be very true more than ever. Security often could mean worst performance and worst user experience so in order to get it right you really need to think about the designs before jumping into to code and consider performance and user experience has main requirements.  I was thinking about writing about how to do HMAC in Java for a while and recently Redis 6.0.0 come out and to my surprise, there was a refactoring on the password part in order to use HMAC.  Security easily could scare engineers often because it is not something well spread yet but I believe this will change soon. It does not matter if you have to deal with PII Data or not, security is super relevant because everybody is running their workloads at the cloud or with IoT and Edge devices which means more distribution, more code, more points of fail
read more »