Diego Pacheco Tech blog

Estimates are lies, estimates are not reliable. It's super common to have companies doing estimates completely wrong. There is no way to do estimates right, however there are ways they can be even worse than usual like: A) Having managers doing estimating for engineers. B) Not accounting holidays, Refactorings and Bugs. C) Not comparing working items D) Ignoring variation: Different sizes, complexities, technologies, needs. E) Not understand when the system is stable or not. Very often estimates are done before the project or product start and where variation and uncertain is sky high. When we take a look into PMI, which is based on bridges and buildings we still see buildings and bridges with extreme delays, extreme over budget and poo quality. If this is true for construction engineering why the hell would not be true for software engineering where we barely have 50 years of experience as a industry compared with the +2K years in construction engineering. Project after pr...

Apache Kafka  is Open Source Streaming platform for streaming and also a distributed log cluster. Kafka makes sense In lots of scenarios like Microservices, Analytics, and Machine Learning, Observability and Logs Injection, Click Stream Analytics and even for the non-traditional messaging platform. 

I'm working with Agile methods since 2006. In this last +10 years, I learned a lot about Agile Principles, Lean Principles, DevOps, Lean Startup, Lean Discovery, and much more. Today I want to share 4 pieces of training I did from 2010 to 2013 which I compressed in 2 videos. In these videos, you will see agile principles, lean principles, coaching techniques, agile coaching techniques, and tools and also experiences and lessons learned in this +10 years working with engineering teams. Fortunately or unfortunately the videos are all in Portuguese, the slides are in English but the videos dont :(.  I hope you guys enjoy and have fun and hopefully learn something that you can apply in your time. These videos are focused on for those who are tech leaders, managers and agile coachs who want to learn more and want to know how I operate with my teams.

Github is the best platform for remote work nowadays. I'm not saying that because my company( ilegra ) is a Github partner but because I deeply believe that. I always liked Github, in fact, way before my company become a partner. Anyone can use Github anyway, however just dropping code in Github won't do the trick, there are patterns and practices you can follow to the best out of Github. PR's(Pull Requests) can be great or can be a nightmare depending on how you do. I use Github since 2009(Yes, +10 years), today I want to share the trick and patterns I use working with Github with my teams in remote projects. There is no right or wrong, this is what works for me, I'm not saying this is the only way, but here we go on what I'm doing so far. This is not the first post I do about Code Reviews, this is more towards remote work but you won't check these posts out: 1. Lessons Learned and Experiences doing Code Review. 2. Business Review: The Gateway to Automati...

For more than 10+ years I run coaching sessions with the folks that report to me. There was a time where I had more than +30 direct reports and other times I got 3 direct reports. Currently, I have 8 direct reports to me (+40 indirectly) which I believe is almost the ideal number. IMHO 5 is the ideal number.  IF you run 1h coaching session which 8 people you still can do all your coaching sessions in a single day per week. However, if you do have 5 you can do with one afternoon with is much better and also you can pay much more attention to the people you are working with.  People come and go all the time, time to time people change life priorities and projects but as much as you can do 101s properly is crucial to improve your relationship with people not only to mention the support and speed you can provide to their career and direct benefits for your day by day projects and endeavors. Coaching sessions don't work alone, they work best with regular retrospectives and Open H...

So I thought several times if I should or should not write this blog post. First of all, I did not want all to make this post a kind of complaint. Companies have structures and roles. People tend to FIT people in the structure they have which makes all the sense. So let's say you want to build a boat, so you better like engineering and water, right? If you want to build better boats them you have 2 paths pretty much, first, you can know more about boats and get deeply involved in every aspect of the boat and therefore meaning being more technical. There is an extremely technical and deep example today of a company that does that which is Amazon. Amazon is extremely successful not only because of that but for sure they pay right attention to technical detail and really know what they are doing.  However, that's not the only path.

This is the second post on Linux Terminal good. The first post was more successful then I could possibly imagine. Thanks to an HN Thread I got some great feedback and decided to create a second post.  I was surprised to see how much people care about the terminal and how much impact the first post made. I also learn some cool stuff from the Thread and want to show it here.  I'm using ubuntu 18.04 LTS with ZSH using OhMyZsh .  For this post, I will show 5 utilities to make any engineer life better.  If you want to learn more on OhMyZSH check this one . If you like vintage terminals also check this one out . So So the first 2 tips will be for ZSH but the other 3 will work out for BASH too. let's get started. :D

Rust does not stop for one second to amaze me. It looks like Rust was designed to talk to every other language so easily. No kidding, it's clear way Web Assembly and rust have lots in common. Java is never an easy lang to interop with. Today I will show how easily we can interop Java & Rust. In order to do so, we will need to use the Java and Rust compilers. For Java, we will not require any other lib than the standard JDK. For Rust, we will need to create a lib project in cargo and we will need to use the crate JNI. I love this feature because java does not allow low-level code on the main language. So if we need to do something low level, efficiently Rust is my first choice, not only for safety reasons but because is really fast.  Let's get started.

The "analyst" is a very old role. I really think the tradicional role does not make sense anymore, we should see analyst as a discipline/skillset and not as a role. If the last 20 years of software engineer teach us something is that engineers are in the best position to work on proper solutions, however, someone needs to worry and focus on the business problem. IF you are in an Engineering Organization, engineers are great as POS / BAS but if you are dealing with delivery discipline and working with non-it people, engineers might not have the best set of skills to do the business analyst part. However, there are many things BAs and even Designers(UX) can learn from engineers. For this blog post, I will like to focus on the skills and the transformation BAs need to face today to keep up with current market needs, innovation and competition in order to be ahead of the game.

Rust is a very exciting language. Not only for system programming but also for microservices and business development. Today I will show how we can create a simple service application in rust using Iron Framework . For this blog post, we will build a rust binary and run both in Docker and Kubernetes. In order to run kubernetes locally, I will be using minikube but you can use any kind of kubernetes cluster or distribution and it should work just fine. The most amazing thing is that we can build a rust executable with only 3.5MB and a docker image with the only 3.63MB . This is super lean!

Automation is a must. Not only because it's the main principle for REAL DevOps but also because is the only way to work properly on the Cloud. As you move your workloads to the cloud, you will learn you will need to change your mindsets too. Automation is something everybody talks about at the end of the day people dont create 100% automation overnight. Often people dont know what to automate and that a problem with often is not tackle by teams. Today I want to share a kanban practice that I do since 2009 and hope this help your team to increase automation. This practice is called business review.  Most modern teams dont have QA anymore and engineers are responsible for creating their own tests, however often as engineers, we could get sloppy and forget to do proper testing and this sloppiness has a price of bugs created by the lack of tests. Business Review is a peer review process where engineers review their peers work, similar to the Code Review but looking to different thing...

Some time ago I blogged about some cool terminal plugins I was using in my Linux notebook. Also some awesome retro- emulator terminals just for fun :D. Often folks asked me about some of the plugins I still use, so I decide to share some awesome, productive tools I use. Most of the tools I use are individual and isolated but the auto-suggestions, this is based on ZSH and if you are using bash won't work out, the others "binaries" will work just fine since they are not attached to ZSH. IMHO if you use bash you should give ZSH a shot because is amazing and has an active community with some many cool and productive add-ons.

First of all, I want to explain why I added a puzzle as an image for this post. This is a metaphor I'm using with several people I work. People sometimes get lost and cannot keep with modern software initiatives. Well, agile is saying that for ages and people still don't get it. There is a huge difference between Simple and Complex systems. More and more making a digital product is from far to be a simple task. However, we still want answers that we cannot get. Sometimes this creates anxiety and we need to learn how to deal with failures, chaos, bad feedback and even that our solution my suck. Ouch thats very dark, right? No, not really, this is the reality, the REAL WORLD is a cruel place and ADULTS need to call tough decisions every day. So what's the deal with simple linear systems? well they imply that quickly you can understand what you are doing and make predictions about the outcomes. Well, digital products are not like that, we cant make predictions on simple line...

Product culture and mindset is one of the most important shifts enterprise companies need to go through. Project mindset has several flaws and yet we still follow it and thats need to change.  Project culture makes us focus on the wrong outcome like Dates, Escope, Feature, Cost instead of value and User Obsession. Since 2014 I follow Allan kelly work and read his books and saw several of his videos. I also exchange some emails with him back on the days. I was amazed how makes sense to back into 2014. Even almost 5 years passed and companies still don't get it. There are other movements that push to the same direction as NoEstimates , Beyond Budget ,  OKRs , Modern Agile ,  LeanStartUp . All these movements / Books have great synergy and they basically make you thing software development in a completely different way.

Hey, guys, I'm here to share some of my experience around principles of Agile and Lean for software development. I'm fully aware that agile is long dead. However besides all craziness and mgmt dogma IMHO there are still value somewhere. Unfortunately or fortunate these ideas were recorded in Portuguese. The decks are in English but the language is Portuguese. Hope you guys like it and learn some different point of view/perspective.

Learning is a chaotic process sometimes. It's like trying to fix a puzzle with 1k pieces, sometimes you don't know where you are going but you need to feel the vide and let your gut feeling drive you. Being a software engineer means that I need to learn every single day, otherwise I'm devaluing myself and also devaluing the solutions I built and therefore the customer is getting less value at the end of the day. IMHO a software engineer is like a startup that went IPO if it's sexy, healthy, awesome and doing bleeding edge stuff the stocks go up otherwise they might go down. However, for many engineers, the only way to get better in change jobs which I think is ultimately true in the IT industry, however, I would like to ask a question, What's the point to change job if you will be the same? So in order to change you need to change first. Rust language was the last language I learned so far(2019). Every time I learn a language I learn something new and that's ...

This is the 3rd part and last one on the series: Reflections on Serverless from SOA to Serverless. On the previous parts 1 and 2 I was explaining how software architecture evolved in several movements and how much we need to change our mindset in the sense of Culture, Movements, and Experiments. Our tools are changing pretty quickly and multi/poly cloud will be soon a reality for most companies. Even if so few companies have this problem in sense of availability and active-active global biz, most of the companies can benefit from multi-poly cloud in sense of getting lock in free from cloud vendors, cost reduction for better and multi offerings, better solutions for specific problems and much much more.

Microk8s is another lightweight k8s distribution - perfect for local tests, an interesting alternative to Minikube , k3s , and Kind.  Microk8s works smoothly in 42 flavors of Linux(Geek Moment: You know is the right answer when you see the number 42 . LOL. ).  Microk8s has easy install and several interesting features like Local Storage, Dashboards, Metrics, DNS, Ingress, Istio and much more. Zero Ops needed with a single k8s cluster done right. That's the marketing of microk8s. In practices, the product works very well but I found a bit slow(start and stop). However installing istio, is 1 command line away from happiness.  Microk8s supports several versions of Kubernetes from 1.10+ to 1.13+ right now(20/04/2019) - k8s is 1.14+.  If you are a hardcore Linux user you must try mcirok8s. Running Microk8s on Ubuntu Cheers, Diego Pacheco

Kind - Allow us to run Kubernetes in Docker. This is not a production-ready solution - however, has a lot of potential in order to make a safe and lightweight alternative option to Minikube.  Kind is built with Go and uses Docker API in order to get things spinning. I got impressed with the speed thing runs with kind, this is an interesting alternative together with k3s and Minikube .

Kubernetes is the new Linux. K8s is the spec for the multi-poly cloud world. Running k8s could be very resource intensive, so is always a good idea being able to run things locally. For several reasons like Engineering Productivity, Tests and Experiments and so on and on. If you are working with Istio like I'm, you might realize it's a bit heavy to run local, especially if you do have other things running on k8s.   Minikube is the goto solution for local kubernetes clusters. However, as I said before, it can get pretty heavy when Istio gets involved. So the solution is pretty simple, it not much advertised. Minikube has a profile feature which allows you to create multiple profiles. Each profile will be a 2GB DISK VM created in Virtualbox. This is great because now you can run multiple kubernetes versions and multiple clusters doing multi experiments. IMHO is always great to have a k8s cluster ready to test things so I have multiple profiles like istio, lightweight, tests, ...

We all need information. There are some many things we need to remember on a standard DevOps project. There are multiple things that we can forget pretty easy. I know so many teams that really only on Slack to track information and that is a big problem. IMHO no one wants complex and heavyweight tools to track information. Developers want simple, easy and pain-free tools like wikis. The problem with most of the wiki tools is that they are heavy and complex to manage. IMHO for small and medium projects we a better solution. I love Git and I love GitHub. GIT has some many nice properties like Easy to audit, easy to merge, we never lot updates, we know everything that happened, we can get back in time pretty easy. For those reasons I really like GIT based Wikies like Gollum . Gollum is built with Ruby and has a nice and clean UI. Using Gollum UI - Gollum translates all the changes into git commands and make commits for us. When we run Gollum, it takes care of everything - the only thing...

Kafka is the leading distributed Log OSS solution. It is great to inject large amounts of data and is used by many companies nowadays. Kafka is used in many use cases like Data Injection, Stream/Batch processing, Microservices CQRS, and ES and many others. Some years ago a showed how to get started with Kafka . Now I want to share a pretty useful and simple tool called Kaf . Kaf is written in Go language and was inspired by Docker and Kubectl. Today I will how to download and use Kaf. Have fun.

JQ is a very powerful json processing tool. JQ is great because is written in C and is super fast. JQ also make json appear with a color highlight when you use it in Linux terminal. JQ is useful because you can combine with other DevOps tools like kubectl, docker, ectd, vault, and so on one. Basically, anything that returns a json output you can use JQ in order to process it. JQ has lots of interesting features and is possible to do pretty simple and yet powerful json processing. JQ allow us to subtract parts of the json document and also provide some powerful processing function like map, add, flatten and much more. It's possible to do some very high level and simple functional programming with it. Now let's take a look on Sheet Cheat I made for JQ.

Keys is the default oss standard for K/V store for many use cases and solutions. When you need run low latency, high throughput IN Memory Persistence - Dynomite is your solution. I always wanted to have multi-thread support in redis like Memcached has. Now someone made it and call it KeyDB . The best thing is the fact that this is done supporting the redis RESP protocol, so all your tools, scripts and code using redis protocol directly still works. Another killer feature in KeyDB is the support for FLASH storage, only available on Redis Enterprise. KeyDB also supports backup to S3. For this post, I will show how to build and run KeyDB which is dead simple and also how to use it with Dynomite like it was Redis. KeyDB Benchmarks As you can see on the KeyDB benchmarks, KeyDB does 1.5x less latency and 50% more throughput . These numbers are pretty impressive. The best part is the fact that is all transparent. Running KeyDB Running KeyDB with Dynomite See the Results...

AWS Open Distro for ES is an open source distribution from Amazon for ElasticSearch cluster and Kibana. Today I will show a simple script I made in order to run ES/Kibana locally without pain. You will be able to run the solution without worrying about any configuration. You need to keep in mind this is a developer sandbox script, this is not production ready ES cluster config. AWS ES Open Distro is enterprise-grade because it has advanced capabilities like SQL, Alerting and cluster diagnostics, I like the cluster tools a lot. The Script The script uses docker and link 2 containers(es and kibana). The script also creates an index in ES called Twitter having 3 documents getting indexed with the following props: user, post_data and message.  The script copy a custom kibana.yml file into kibana container in order to pass proper docker DNS / Container link to ES container - by default config looks up to localhost and I have to change to es DNS(same name used on docker link). ...

GraalVM is a universal VM mainly for Java and JVM supported languages. With GraalVM you can run native programs written in Python, Ruy, R and all JVM languages like Scala, Clojure, Groovy and Kotlin also LLVM based like C and C++. Graal VM removes isolation between languages and enables interoperability with natively shared runtime. You might be wondering why botter use other VM? Well, there are a couple of good reasons. Java is a pretty old language and does not have the best startup time. Today we are all running services in containers on top of kubernetes and we are always looking for ways to improve the customer experience. Startup time is not only a matter of better customer experience but also availability. For the developers, this means better development experience and idiomatic experience with the best solution for each specific problem. GrralVM Supported Runtimes GraalVM can be used standalone runtime or embedded in the context of OpenJDK, Node.JS, Oracle...

In the previous blog post , I describe the architecture evolution from SOA to Serverless. For this blog post, I will continue to talk about the same subject in the sense of strategy evolution and architecture strategy. Business value does not come only by adopting new technology but also from a new way of thinking or the right mindset shift and the right understanding about tech culture and vision. Most companies have the right Tech view and often don't see themselves as tech companies because they think this business is logistics, sales, health care, insurance, finance anything but tech. Nowadays several companies depend so much on Technology in order to service, prosper and deliver better customer experience(CX) . However, tech is often seen is a COST CENTER rather than a VALUE center. Because often IT fail to DELIVER and SHOW value to the business. That's why some many companies are looking into the DevOps movement and other movements like Digital Transform...

New technology is great. It's very important to open space for new ideas, better solutions, cheaper, faster and exciting new things. However, even with technology, there are principles and lessons learned with previous technologies we need to carry on otherwise we will keep making the same mistakes and we might not take full advantage of new tech. Why? Because often new tech requires a new way of things and often times also require to keep some things from the previous movements. Companies want to do Serverless now a day, often because cloud vendors are pushing that hard and because of the short term cost benefit. However, in the long run, Serverless solutions might be way more expensive than in-house solutions using OSS. There is no silver bullet, you need to understand what you are doing in order to take full benefit from new approaches, techs, and ideas rather than just moving everything from on-trend to another.  I want to share some thoughts from several architecture...

Kubernetes is becoming the truly cloud-native specification for the cloud. However not all software runs on the cloud, there are other uses cases like embedded software(IoT and microcontrollers) besides IoT there is another kind of computation that is growing more a more nowadays called Edge, thanks to the low latency and facility capabilities. Would be great run the same infrastructure for the edge/IoT as we do for the cloud? Well, rancher folks addressed that with k3s . A super lightweight kubernetes distribution where they replace things and remove features instead of adding, This is great not only for the IoT/Edge uses cases but also for folks who study the cluster in the academia and for the developers in 2 fronts. First on the CI/CD front, so we can spin up a lightweight cluster to run all kinds of tests, this also has a perfect fit with GitOPS model.