DevOps it's about Design

571px-Popular_Science_Dec_1918_p23_-_Ship_Emergency_Steam_Cutoff_Valves.JPG
DevOps it's about Design!

As Steve Jobs once said: "Design is not how it looks but how it works". DevOps implies operation work through automation. People who do DevOps could come from 2 different kinds of backgrounds, you can come from Dev or from Ops. A good Developer cares about and really knows how to design applications. We do have several principles to design software. One of the basic and most important principles is Abstraction.




Abstractions
Abstraction means doing more with less and hiding complexity. It's easy to find automation solution with does not provide any abstraction at all. There is an easy way to get this smell which is the number of parameters you get from a Jenkins job for instance. More parameters means throwing complexity in your face it does not imply more flexibility.


Having more options also can create more trouble for you because. Let's say you are using AWS and you receive the AMI_ID as parameter. Sounds like a good idea but then you realize you have microservices and need to do this for all your 1000 microservices and them there is a new version. Well you need to update all your jobs and make sure to re-deploy each one.


Code generation is not evil but it's very easy to become evil. Better than generate parameters is abstract parameters and as less parameters you ask the developer / engineering to pass the better. So this imply we should have static configs? Not really. We should have dynamic configs and there are several ways to do it so, for instance you could be using: etcd, redis or any other key/value database. It's better to have replicated multi-region, multi-datacenter solution like etcd or even Netflix/Dynomite is you have such requirement.


Telemetry of the Telemetry
Doing so you still have do dynamic configuration but you won't expose complexity to the cloud deployment jobs. All automation work can fail. So you need to make sure you have proper error handler and telemetry in place. Few days ago my car turned on a LED for ABS brake system. After few days I went to the car dealer in order to perform a review. I was told that the LED was for a sensor wih tells you how bad the break pads are. In my case that sensor stop working so we got an LED  notification. This does not means I don't have break or that my brake system crash it only tells me that the brake sensor stop working. I found this very interesting because in the end of the day this is the Telemetry of the Telemetry. So I have a monitor that monitors the monitor. It's kind of meta and inception-ish but IS your car have such system why your complex cloud solution should not have this?
The correlation between design and telemetry is this. You need to think about error cases while you do design. You can't consider only happy scenarios. Failure happens on DevOps Engineering and you need to code solution that address this concerns.
Cloud Deployments is hard
Microservices deploys are kind of easy however when you consider a cloud environment it gets more complicated. Since you might have specialized data stores for each microservice. Like one microservice can use Cassandra and other might use Elasticsearch. Provisioning data stores are often way harder them just the application(microservice). Because you will might need to feed / migrate data and schemas.
Data stores also have their own concerns like backups, restores, warm-ups, cluster management. Automation for databases in cloud it's more complex. In the end of the day it's all about requirements and the use cases you might have. Having more availability makes everything more complex and requires more work.


This kind of automation is very hard to with bash or python scripts for instance. People tend to you solutions in Go or Java for such tasks. Since you have a procedural nature in this kind of work. This is also design.


Trade-Offs come to play
Today we have more than one way to do things. For instance you can use Kubernetes. Which will give you most of this things for free but them you will be coupled to Kubernetes. You also can create your engineering solution and use open source tools which will require more automation work but you will definitely have more flexibility and you will be able to run microservices without docker. Having the possibility to run kubernetes everywhere is sexy and very attractive to me since you could run multi-cloud solution. In order to run your code in other cloud and make cloud deployments easier you also could use Netflix/Spinnaker. Spinnaker it's a multi-cloud orchestration solution.


This is design as well. What you will use it and why? What requirements do you have and what solution address this concerns. Spinnaker can call as many Jenkins jobs as you have so you can see it as one thing which makes your life better but this does not fix your job complexity problem in Jenkins for instance.
Design is a critical discipline in DevOps. It's important in all levels from a python script to a spinnaker pipeline. Having more parameters does not necessary gives you more flexibility. Hiding details is always a good idea. Using CONVENTIONS and Dynamic configuration it's also the way to go.


Cheers,
Diego Pacheco

Popular posts from this blog

C Unit Testing with Check

Image
When I start on IT, several years ago(+15), C was the first language that I learned. But was not the first language I mastered and deployed in production which was...surprisingly Visual Basic 6.0. However, when I learned C or any language back on that old and arcane time noddy was into testing. However now is impossible to work professionally in any language without tests. C is a low-level language but still kicks ass and has several interesting libs/frameworks. People use to fear C a lot because of lack of syntactical sugar but once you get used to it is not that hard at all. Today I want to show how you can do Unit Testing with C. I will not show how to do Unit Testing with C++ but with C ANSI actually. We will be using Check a unit test lib for C ANSI.
read more »

Having fun with Zig Language

Image
Zig is a general-purpose language created in 2016 by Andrew Kelly .  Zig aims to fix the issues C language has while being pragmatic and simple at the same time. Zig really shines as being simple by not having any hidden control flows, not having hidden memory allocations, no pre-processors, and no macros. Basically, it's one language and one language only. Since pre-processors and macros can get really complex and become their own universe. Sure it's not the Zig Philosophy.  Zig Compiler(Zig cc) also is a C compiler, and the interoperability from C to Zig is very good. Clearly, we can see influence from other languages like Go and Rust; however, Zig is pretty unique and different from Go and Rust.  Zig is fast. Zig is perfect for system programming. I bet we will see a lot of solutions being built in Zig. As a languages aficionado, I like to learn at least one language per year and sometimes more; Zig was my 2021 pick, did not regret it. Even if you dont use all language...
Keep reading

HMAC in Java

Image
In 2018 AWS CTO(Vogels) said : "Security is everyone's job". For the last 2 years, there was so many famous and big data leaks and breaks that made that statement be very true more than ever. Security often could mean worst performance and worst user experience so in order to get it right you really need to think about the designs before jumping into to code and consider performance and user experience has main requirements.  I was thinking about writing about how to do HMAC in Java for a while and recently Redis 6.0.0 come out and to my surprise, there was a refactoring on the password part in order to use HMAC.  Security easily could scare engineers often because it is not something well spread yet but I believe this will change soon. It does not matter if you have to deal with PII Data or not, security is super relevant because everybody is running their workloads at the cloud or with IoT and Edge devices which means more distribution, more code, more points of fail...
read more »