Posts

Experiences Building a Cassandra Orchestrator(CM)

Image
Cassandra is a kick-ass NoSQL database. Battle tested by Netflix, Apple, Uber and so many great cloud scale companies. If you want to run Cassandra in production you need to buy DataStax Enterprise or engineer your own solution - since Cassandra community is not enough. My company loves open source and my customers love open as well so we decided to build our own Cassandra orchestrator for a lack of better name we called it CM(Cassandra Manager).  CM runs on AWS(Ec2) and managed single and multi-region clusters for Cassandra. CM has java interfaces so even today it just runs on AWS it could easily be ported to in containers or other fabric runtimes, which may happen on the long-term future. So it might sound crazy when you think about to build engineering around Cassandra but is actually not several companies in Silicon Valey like Netflix and outside of the valley do similar things or Databases being NoSQL or Relational. Automation is must if you want to scale, automation is way more …

Writing a simple REPL in Go

Image
Go is a great language for writing tooling. It's very easy to produce a self-contained binary in go which makes very easy to distributed tools written in go. Today I want to show how easy is to write a simple REPL in Go. REPLs are quite popular in Functional Programming community like Scala, Clojure, and Haskell. Today even Java has a REPL :-). Cassandra has interesting REPL called CQLSH which allow local and remote connections. Currently, I'm thinking to write REPLs for some things I´m working with. REPLs are great for productivity and context. No to mention the could reduce and simplify discovery and troubleshooting. You might be thinking Gosh to write a REPL is tough we need to have parsers, print things nicely with colors, use tab completion, remember all previous commands. and one and on... I'm a full fledge REPL might require more work however a basic and simple version is something we can do in less them 5 minutes so I won't be doing any complicated parser or p…

Lessons Learned and Experiences doing Code Review

Image
Today I want to share some experiences and findings in regards to Code Review. Code Review is a practice that everybody does it today. I was doing code review way before Github existed. I will share experiences before Github and after Github. Difference companies and different teams have different realities and preferences. I don't think there is universal or wrong but this is my experiences. I never was and still not a big fan of tools for code review. This subject is very related to Technical Debt. I think there are 2 kinds of technical debt the one you pay often and the ones you need to schedule because they are big and might need lots of time to fix it. However, if you just managed tech debts often means you never pay it. Also, you should be paying even if no one outside of the team is asking for it. When you are doing a PR is a good time to pay some debts related to that PR, also long as is not too much. I found Github view for code review very limiting. Small config changes…

Having fun with Cats

Image
Cats is a library for Functional Programming in Scala. The name come from the Inspiration by the Category Theory so Cats is short for Category Theory. Cats are about types, Type System and having abstractions to make it easy to work with them. If you want to go to the next level of function programming Cats is the way to go, however, I need to say that it's hard, get the ideas to take time, understand the terms, principles, and applications takes times as well. At the end of the day, this is about style so there is no right or wrong. Recently I made a post about Monads so it might be useful for you to read it if you did not read it yet. This post won't be monad heavy but reading about monads will make it easy to understand some things here. It's fine if you don't understand everything, I'm 100% sure I don't understand everything and there are lots of ideas and concepts that I still learning. So this takes time. I don't think this is something will make sens…

Scala Monads 101

Image
What is a monad? It's easy to read lots of sites and still don't get the idea. Many people already used monads but few really understand them truly. Monads are a kind of a Myth in Scala for lots of developers. ItÅ› possible to code without creating your own monads? Yes for sure it is. It's very likely you already create abstractions which were similar to monads but with different names. Here are some samples of Monads in Scala: Try, Either and Option. We have some of this monads in Java 8 as well like Optional. Monads come from the Category Theory which is a branch from Math. Monads are very strong concepts in Haskell Language for instance. IMHO one of the reasons why use Monads rather than your uncle abstraction is that they are universal and in theory, they are easier to understand, in practice if you don't study math, Category Theory and Functional Programming principles actually monads will sound much harder than you user uncle abstractions. So, in other words, we …

Experiences Building & Running a Stress Test / Chaos Platform

Image
Stress Test is something that everyone should be doing if you care about performance. I run software at AWS so is easy to spin up more boxes but is also more expensive and you might be in a trap because you might have ineffective software with Memory Leaks, Hidden Bugs, and untuned servers which will all show up with scale or stress tests - whatever happens first. Unfortunately, this still not quite popular among most of the microservices developers. Create and run stress tests is something hard and tedious because it involves lots of moving parts. A single microservice could call several other microservices some of the downstream dependency graphs could get complex easily, this is just one of the many challenges involved with stress tests. Today I want to share some experiences in building and running a stress test and chaos platform. Do you know how many concurrent users or requests per second(RPS) your microservices can handle? Do you know what it will be the COST to scale your se…

github-fecther: Checking new Repos from Github with Go

Image
Go lang is a very interesting language. Github already is the central hub for the open source on the Internet. Companys have lots of repositories and it's very hard to catch up nowadays. So I build a simple silly tool in Go in order to fetch all repositories a company might have and they compare with the previous fetch and see if there are new repositories.  Thanks to GO elegance I was able to do that with less than 200 lines of code. In order to do what we need to do, we will use some basic functionality in Go such as IO, Http Calls, JSON support for Structs, Slices, Maps, Error Handling(Which is the sucking part) and a bit of logic. This program can fail since I'm not using a proper TOKEN for github so you will HIT the github Throttling limits pretty quickly if you keep running the program. You can create a proper DEV account and fix the code to use your token or you can schedule the program to run in different times in your crontab. This is not production ready, but it'…