Diego Pacheco Tech blog

Lettuce is a very efficient and extensible Redis client for Java.  Lettuce is interesting for many reasons but one of the big ones is the fact that provides an Observable / Flux API. So you can very efficient reactive programming with RxJava or Reactive-Streams power by Reactor . I'm using lettuce in production in the last 2 years and have been really great. For this blog post, I will how to dispatch custom commands to redis-modules via Lettuce.

Cthulhu is an interesting project. It allows you to code custom redis-modules in JavaScript . This is very interesting because JS is very productive and easy to write code. You might be wondering how they made this efficient? Well, the secret is a project called Duktape which is an efficient JS engine written in C. So Cthulhu + Duktape will pre-compile your javascript function and run it on C which will be efficient and will be able to talk with redis core. Unfortunately, not all functions are available -- you can check the ones that are available here: List of available APIS . You can so a simple comparison with a module In C I wrote some time ago in this blog post . C is way more powerful but also way more error-prone and might take more time to write if you are not used to it. So for this Blog Post, I will how to download and compile Cthulhu and they install in Redis 4 and code some custom function in JavaScript. Let's get started.

Some time ago I posted about some lessons learned by doing DevOps engineering and in the beginning of the year, I was posting some ideas and experiences(good and bad ones) around  Design and DevOps Engineering. I do lots of different things on my daily basis like Architecture, Software Engineering, DevOps Engineering, Chaos / Stress Testing, Reliability Engineering, Consultancy, Management / Coaching.  I'm lucky to run a team of great architects and also code every single day,. This provides me great insights since I'm executing things but also reviewing things other people do. There are lots of things I think every day but more and more I think about Design and DevOps Engineering. The right designs and how things get bad and complicated by the lack of good design. There are several elements of good design. I already cover some aspects in other posts and today I will cover other things like Assumptions, The Right Language for the job, Multi X Single language,...

Akka-Streams is one of the many interesting and very useful Akka modules. Akka is a powerful actor / reactive framework for the JVM. Akka is an extremely high-performance library -- you can do Up to 50 million msg/sec on a single machine. Small memory footprint; ~2.5 million actors per GB of the heap. Akka is also resilient by Design and follows the principles of the Reactive Manifesto . Akka follows Erlang Actor philosophy/ideas.  Many big and successful companies use Akka in production like Blizzard, Intel, Wallmart, Paypal, Amazon, Zalando, Netflix, IGN, VMware UBS, and much many more others.

Cassandra is a Rock solid AP NoSQL database. For this blog post, I will share a simple recipe to deploy a Cassandra cluster on AWS/EC2. There is the DataStax AMI or  DSE you should consider for production workloads.  This recipe I'm sharing is for Amazon Linux (CentOS based) but you can do for Ubuntu or even in Docker if you want to. Keep in mind this is for Development / Experimentation purpose. I'm not covering proper tunning for your workload, Compaction Strategy and Keyspace design here and you also should be doing this under multiples ASG 1 per AZ ideally. So Let's get started.

Dyno-Queue is an interesting queue solution on top of Dynomite . Dyno-queue uses dynomite java driver a.k.a Dyno . Dyno Queue gets all benefits from dynomite like Strong Consistency, High Availability, Stability, High Throughput and Low latency and extend to queue semantics. I highly recommend you read Netflix post about dyno-queue.

Akka a great actor framework for the JVM.  Akka is high performance with high throughput and low latency, resilient by Design.  It's possible to use Akka with Java however I always used and recommend you use with Scala . Create actors systems is pretty easy however creating multiple nodes on Akka cluster locally could be boring and error-prone. So I want to show how easy is to create an Akka cluster using Scala and Docker (Engine & Docker-Compose). We will create a very simple actor system, we will just log events on the cluster however you can use this as a base to do more complex code.

Scala language it was very inspired by Haskell . Type classes in another sample of this and many inspirations  from Haskell. There are always several ways and styles of thinking to address problems. Type classes is another way of thinking and dealing with problems. For some problems, you might just use Pattern Matcher -- But if you have to add a type? Right, I could stick with good old OO since Scala is hybrid right? --  But what if you need to add one operator? Type Classes can help you out.

Dynomite is a kick-ass project. Basically, allow you to have strong consistency on top of NoSQL Databases. I've been using dynomite for a while in production( AWS ) and I can say the core is rock solid and it just works. Lots of developers use Windows or Mac for instance and dynomite is built in C and it's really meant for Linux(Like all good things).  So some time ago I made 2 simple projects to get started quickly with dynomite.  Basically, the project creates a simple dynomite 3 node cluster and let you run on your local machine with docker . There are 2 projects - One to create a dynomite cluster with Redis -- The other with Facebook's RocksDB (Experimental). So you can use it on your local machine to Debug and it works just fine. So why not go 1 step further and run Dynomite in AWS using docker? There are cool benefits if you do this approach.

DevOps Engineering is about Workflow Currently, I work doing a bunch of different things(like most of Brazilians). I do Architecture, DevOps Engineering, Software Engineering, Agile Coaching and a bit of management. There different types of companies and cultures but basically or you are more specialized or more cross-functional I think in general Brazil is similar like Europe and more cross-functional and the US is more specialized but really depends on the company. I also run a small team doing: Dynomite / DM , Serverless Remediation, Stress Test and Chaos Platform and Telemetry / Observability. My teams need to provide solutions(engineering)but also Stability .

Want to learn how to build microservices using Java 8, NetflixOSS Stack(Eureka, RxNetty, Feign, Hystrix, Ribbon) using Kubernetes(Minikube).  That's the real deal you read it right. Get my videos series: Building Effective Microservices 80% until 30th November 2017.

Redis is a great K/V store written in C. Redis can accomplish a lot for a single thread process. There are many applications for redis like for instance:  - Frontend Database - Real-time Counters - Ad Serving - Message Queue - Geo and TimeSeries DB - Session State - Cache That's all great but let's say I want more, how can we customize redis? There are some options like a Lua Script , Fork Redis, Tak to @Antirez or Create your own NoSQL database based on redis or not. Redis provided another solution - Redis added support for external modules in 2016.

NetflixOSS Archius 2 is a great Dynamic Configuration solution for microservices. Archaius is based on Apache Commons Configurations project . Using Archaius we can load configurations from several sources such as OS env vars or any Database like Oracle or even from Zookeeper. If there is a missing configuration source you can add it pretty easy and load your configs.  Archaius can be used in any java project no matter if is a microservice or not.  Archaius also support dynamic configuration refresh via callbacks -- In short, this means you can reload your configs without re-deploy or downtime in your microservice -- which is really great. Archaius has many nice features. Archaius also is very modular and easy to extend there are some nice community extensions available as well. 

NetflixOSS Governator is a set of Google Guice extensions to create REST services using Jersey . Using Governator we can easily configure servers like Jetty and Tomcat in order to build microservices. We also can use set of guice modules to integrated with Archaius and Eureka-Client. Governator is not opinionated, it's similar to Spring Boot in comparison. However, Governator is configured to work with Guice and not Spring framework. Governator it's cool because you can define pretty much everything using java code and annotations in a declarative fashion. All code is configured in Guice so we can take benefit of Ioc and Dependency injection and end up creating solution more testable by nature.

DevOps Monolith The term Monolith it's often used in legacy systems or system poorly designed and poorly architected.  Microservices are the facto stand architecture when we talk about software. There are lots of companies also doing DevOps Engineering and DevOps per se its related to microservices. Microservices required some infrastructure work like for instance: Provisioning : Install OS system packages, files, scripts. Telemetry : Dashboards and alerts for you microservice eco-system. Testing : Stress Testing, Chaos Testing, Load Testing, etc... Canary : Automated canary analysis, deploy, score, and rollback. Infrastructure work could be more or less depending on your level of abstractions for instance if you are working with bare metal or IaaS you will definitely have more things to get done. However, if you are doing cloud-native microservices using kubernetes or any FaaS stack you might have less work but always well have some infrastructure wor...

Ano passado eu escrevi o meu primeiro livro. Depois de 3 meses de trabalho consegui terminar minha primeira serie de videos. Obrigado Deus por mais esta vitória e agradeço o apoio de todos. Last year I wrote my first book. After 3 months of hard work, I managed to deliver my first video series. Thank God for this big Win and I thank all the support I got. Building Effective Microservices. Videos: https://www.packtpub.com/…/building-effective-microservices… Codigo: https://github.com/diegopa…/Building_Effective_Microservices Cheers, Diego Pacheco

Lessons Learned with DevOps Engineering Experiences Almost 20 years ago an Agile movement was born. After 20 years there are still lots of confusion and misunderstanding on principles, practices, and mindsets. DevOps barely have 10 years but does not have a method like a scrum, Kanban or XP. However, DevOps rely on Lean / Agile there is no strict definition, method or formal guidance how to do proper DevOps.   Like everything in life, this is good and bad at same time. DevOps is a movement about experiences that made some for some companies and people in order to have or practice better ways to develop and operate the software. Currently, DevOps is a mess. DevOps is a mess is Brazil but is also a Mess in the USA. For lots of people, DevOps means Ops doing code. For other companies DevOps means Cloud Ops Team doing ops. There are DevOps folks from Development background like me and there are much more. There is so much confusion on the market that you really need...

Dynomite it's a great solution for clustering with NoSQL Databases like Redis . Eureka is a nice Registry & Discoverability Solution. We can get best of both worlds using Prana . Prana is a sidecar that enables non-JVM applications to register in Eureka. Sometimes we could easily use multiple discoverability solutions like DNS, ETCD, Eureka etc... However not all discoverability provide the same benefits some tools are better suited for some jobs them other. I like ETCD and make sense onKubernetes world but if you are doing Java Microservices eureka makes more sense. The triad(Eureka, Prana, Dynomite) is great, This is great because then you can do discoverability on your database nodes, this is not great for several reasons like: Use the same tool for Registry / Discoverability Enable all sorts of dynamic programming which is great for DevOps Engineering Avoid AWS Throttling issues Make dyno clients more dynamic and this is a better solution them DNS like route5...

The Stability Mindset I have background on software development. Currently, I'm leading a team of highly effective engineering's where we are responsible of some of the components we delivery in production and I need to say that is not the standard reality on it market. Development is used to break things and move faster, however, lots of developers never cared about production or about “polishing” they systems. This effect happens some several reasons like: Having different people to develop and operate software Having different departments for development and Operations Having development teams focus too much in new features rather than real UX So there is this culture clash when we have developers doing DevOps Engineering(I say that for my own experience). We developers are not used to keep something running no matter what. To be always available and reliable. It takes some time to get this “Stability Mindset” but once you get it is hard to get out of you...

Tests are at same time a simple and a complex subject. Testing is something basic. Are you don't do testing how do you know it works? How do you know your software won't stop working after some refactorings?. Unit Tests are the basic level of testing you could do.  In the past unit test was normal. When we deliver software we need to deliver unit tests whiten so level of coverage. When we talk about coverage things can get tricky since you might end up coding tests for parts of your application where it is little or no value what so ever. Coverage can be something questionable as you language might force you to do less or more tests. For instance is you have a strongly typed language where you enforce as much thing as you can via compiler you might need to do less testing since the compiler works in your favor.  As you work with more dynamic and weak-typed languages you might need to do more tests since the compiler might catch fewer things. As much the language e...

All companies want to innovate currently today not everyone can actually. There are several bottlenecks for innovation. The technology it's often part of the problem and part of the solution. Software architect there is a crucial role in enabling solutions that not only scale and are available but also that make the difference. Many companies have monoliths or have several monolith systems and they are having trouble to keep up with the market pace of innovation. The cloud seems to be the right place to go. However, if we don't think how to do it we could be trapped in a problem bigger than we think. Currently, Microservices become a de-facto standard of architecture for several problems however in order to get more benefit from micro services we need do some mindset shifts. I'm not talking about basic shifts such as Independence and Isolation .

Unfortunately, FP is a religious suggest for a lot of people. I jumped on the FP Religious bandwagon for awhile back into 2012. Today I consider myself more practical rather than just "pure" in regards of FP. To be 100% clear my goal is not to be controversial or create any kind of flamewar. I feel free to disagree with me. The fact is I like Scala a lot and is very hard to talk about java 8 without talking about Scala. IMHO Java 8 got some good influence from Scala. IMHO I don't think Java 8 is a drug to get into Haskell :-). Java 8 got some sort of FP influence, Thanks to Scala I would say. I suppose to deliver a talk about functional and reactive programming with Java 8, 2 years ago and I found this slide deck in my machine so I thought about making a video presenting some of the functional programming principles with Java 8 including frameworks such as RxJava. So this is a ~40 minutes video - I hope you guys enjoy and have fun.

Go is a simple, fast and powerful programing language. Go is growing a lot into the DevOps Engineering scene like Hashicorp Stack or even Kubernetes .  One of the main advantages of GO is the fact that you can generate a single binary with all you needed, bundled in a single file. This makes distribution so much easier. Go is also very compact for some use cases and you can write so less code and still very very efficient and get best of performance.  Today we will see how to create a very, very, very simple service in go. This service will access redis to increment how many times it was called.  We will use Minikube in order to run kubernetes locally and we will store our data in Redis.

Kubernetes is getting more popular every day. Kubenertes is an open source system for automating deployments, scaling and managing containerized applications. Created by Google on 2014 and also know as k8s. Why? Because there are 8 letters between k and s :-). K8s has many features such as Automatic bin-packing which is the capability of placing containers based on resources and constraints.  K8s also has horizontal scaling, storage orchestration using local storage or cloud storage such as AWS or GCP. K8s has important Cloud native capabilities such as Self Healing, Service Discovery and Load Balancing and secret and dynamic config management. For this blog post, we will see how to bake a simple docker image using node js application and deploy this docker image on hibernates using minikube in order to run locally. There are other cloud-native solutions such as NetflixOSS Stack . Also Spring Cloud , which uses NetflixOSS too. However, on this post, we will ...

As Developers we spend lots of time using Linux terminal. Today I will show my set of customizations I'm currently using. The terminal setup we will see is awesome because it makes your life easier using git and also display useful information about Linux. I will show how to configure Oh My Zsh with Needfonts, Powerlevel9k theme and ruby colors on Gnome Ubuntu 17.04. Tools that we will use: Oh My Zsh NerdFonts Powerlevel9k theme Ruby Colorls Let's get started! 

Gradle right now is the standard tool for Java builds. Before gradle, pretty much everybody was using maven . Working with cloud computing very often we need do remote debugging. In this blog post, I will show how to configure remote debugging for all your gradle projects.  The nice thing about the approach I will show is that you don't need to change your build.gradle files since we will do it as a global config this will be valid for all. We also will setup configs for Eclipse and IntelliJ IDE support in the same file. Since we will configure remote debug you need to keep in mind that debug port need to be open in all security groups and firewalls in order to work.  Keep in mind remote debug is slow especially with Amazon. We will open port 9009, but you can change the port if you like. This config works with Jetty and also with Java main classes as well.

Dynomite is a kick ass cluster/proxy solution that provides high availability and strong consistent to databases. Dynomite was created and battle tested by Netflix using Memcached and Redis as primary store backend. RocksDB it's a Facebook Embedded Key/Value store which is growing up a lot because of his incredible high performance and low latency. For this blog post, I will show some simple project I create for the developers to get started with Dynomite and RocksDB together. This project uses docker and creates a Dynomite cluster using RocksDB as backend. Let's get started...

I often don't blog about this kind of stuff. Yesterday I friend of mine showed me this awesome project called  cool-retro-term . When you work with Cloud Computing and DevOps Engineering you spend lots of time on the black screen testing things on the cloud.  When I saw this project I got very excited because it brings some nostalgy back to my life. To be 100% clearI'm not that old as my friend :-) However I share his joy into this retro terminals. This also reminds me some old games like Fallout 1 . I just this is pure fun so I'm sharing here with you guys some screenshots and also how to run on Gnome Ubuntu 17.04.

Cassandra is a kick ass NoSQL Database that guarantees strong consistency.  Inspired by Amazon Dynamo paper Cassandra can deliver high availability and strong consistency.  Battle tested by companies like Apple, eBay, Netflix, Github, Nasa and several others. For this blog post, I will show how to setup a simple Cassandra cluster. This setup is for development purpose and it's not ready for production.

I update my Linux from Ubuntu 16.4 LTS to Ubuntu Gnome 17.4 - Now I'm here sharing my experience. Ubuntu Gnome is Ubuntu with Gnome. This is the latest version of Ubuntu Gnome since Canonical announced they are killing Unity - next year Ubuntu will come with Gnome by default.

Quality is such an overrated word. Itś so 80s to say that you are doing something for quality sake. Today quality can appear with a different meaning -which often means specific things which people care about let's say Green or Organic or Oil-free. When we are are talking about software development it's the same thing. Quality is a 2k thing and nobody uses in this way anymore however when you think more most of the times we are talking how we do things like automation or microservices or Chaos Engineering this is what we do and how we do it. It's quality but with a different and specialized focus.

I was wondering if I should post or not my thoughts around TDD. TDD always was and I think it will always be a polemic subject. From time to time someone published something and the never-ending-debating come up again. Uncle Bob post "TDD Harms Architecture" . I do agree with most of Uncle Bob points, however, I do want make clear some of my own opinion on the in this matter.

DevOps it's about Design! As Steve Jobs once said: "Design is not how it looks but how it works". DevOps implies operation work through automation. People who do DevOps could come from 2 different kinds of backgrounds, you can come from Dev or from Ops. A good Developer cares about and really knows how to design applications. We do have several principles to design software. One of the basic and most important principles is Abstraction. Abstractions Abstraction means doing more with less and hiding complexity. It's easy to find automation solution with does not provide any abstraction at all. There is an easy way to get this smell which is the number of parameters you get from a Jenkins job for instance. More parameters means throwing complexity in your face it does not imply more flexibility. Having more options also can create more trouble for you because. Let's say you are using AWS and you receive the AMI_ID as parameter. Sounds like a g...