Diego Pacheco Tech blog

Keys is the default oss standard for K/V store for many use cases and solutions. When you need run low latency, high throughput IN Memory Persistence - Dynomite is your solution. I always wanted to have multi-thread support in redis like Memcached has. Now someone made it and call it KeyDB . The best thing is the fact that this is done supporting the redis RESP protocol, so all your tools, scripts and code using redis protocol directly still works. Another killer feature in KeyDB is the support for FLASH storage, only available on Redis Enterprise. KeyDB also supports backup to S3. For this post, I will show how to build and run KeyDB which is dead simple and also how to use it with Dynomite like it was Redis. KeyDB Benchmarks As you can see on the KeyDB benchmarks, KeyDB does 1.5x less latency and 50% more throughput . These numbers are pretty impressive. The best part is the fact that is all transparent. Running KeyDB Running KeyDB with Dynomite See the Results...

Working with the cloud is great but it's not easy. The Cloud allows us spins machines pretty easily, however, update these machines are not that an easy task, especially if we talking about databases. Today I want to share some experience with remediation process I've built for Dynomite in my current project.  I'm using NetflixOSS Dynomite in production(AWS) for more than 2 years now both as cache but also as Source of Truth. I need to say that is rock solid and just works. However often we need to increase some machine capacity, i.e: Add more memory or increase the disk size. There are times where we need to apply patches on the OS - I use Amazon Linux on Production on EC2 and recently there was the Meltdown and Spectre situation. Other times there are telemetry or even provisioning and configuration bugs that need to be fixed.  No matter the reason you will always need, time to time, change something on the underlying AMI or in the Java or C application which is ru...

Lately, I'm working with custom redis-modules and I need to say: redis-modules kick ass. I started working with Cthulu and coding modules with javascript which was nice and very productive but given some of my current use cases I had to go to C. I recommend you take a look at my previous posts here:   -  Building Redis-Modules with JavaScript using Cthulhu   -  Dispatching Custom Commands with Lettuce and Redis-Modules Today I want to you how to do proper debug with Redis modules written in C language. I will be using eclipse Oxygen CPP in order to debug redis module and redis 4. Here there is a video which I recorded with a simple live demo about how to debug a redis-module project and redis. The source code is available on my GitHub . I hope this is useful for you.

Lettuce is a very efficient and extensible Redis client for Java.  Lettuce is interesting for many reasons but one of the big ones is the fact that provides an Observable / Flux API. So you can very efficient reactive programming with RxJava or Reactive-Streams power by Reactor . I'm using lettuce in production in the last 2 years and have been really great. For this blog post, I will how to dispatch custom commands to redis-modules via Lettuce.

Cthulhu is an interesting project. It allows you to code custom redis-modules in JavaScript . This is very interesting because JS is very productive and easy to write code. You might be wondering how they made this efficient? Well, the secret is a project called Duktape which is an efficient JS engine written in C. So Cthulhu + Duktape will pre-compile your javascript function and run it on C which will be efficient and will be able to talk with redis core. Unfortunately, not all functions are available -- you can check the ones that are available here: List of available APIS . You can so a simple comparison with a module In C I wrote some time ago in this blog post . C is way more powerful but also way more error-prone and might take more time to write if you are not used to it. So for this Blog Post, I will how to download and compile Cthulhu and they install in Redis 4 and code some custom function in JavaScript. Let's get started.

Cassandra is a Rock solid AP NoSQL database. For this blog post, I will share a simple recipe to deploy a Cassandra cluster on AWS/EC2. There is the DataStax AMI or  DSE you should consider for production workloads.  This recipe I'm sharing is for Amazon Linux (CentOS based) but you can do for Ubuntu or even in Docker if you want to. Keep in mind this is for Development / Experimentation purpose. I'm not covering proper tunning for your workload, Compaction Strategy and Keyspace design here and you also should be doing this under multiples ASG 1 per AZ ideally. So Let's get started.

Redis is a great K/V store written in C. Redis can accomplish a lot for a single thread process. There are many applications for redis like for instance:  - Frontend Database - Real-time Counters - Ad Serving - Message Queue - Geo and TimeSeries DB - Session State - Cache That's all great but let's say I want more, how can we customize redis? There are some options like a Lua Script , Fork Redis, Tak to @Antirez or Create your own NoSQL database based on redis or not. Redis provided another solution - Redis added support for external modules in 2016.

Dynomite it's a great solution for clustering with NoSQL Databases like Redis . Eureka is a nice Registry & Discoverability Solution. We can get best of both worlds using Prana . Prana is a sidecar that enables non-JVM applications to register in Eureka. Sometimes we could easily use multiple discoverability solutions like DNS, ETCD, Eureka etc... However not all discoverability provide the same benefits some tools are better suited for some jobs them other. I like ETCD and make sense onKubernetes world but if you are doing Java Microservices eureka makes more sense. The triad(Eureka, Prana, Dynomite) is great, This is great because then you can do discoverability on your database nodes, this is not great for several reasons like: Use the same tool for Registry / Discoverability Enable all sorts of dynamic programming which is great for DevOps Engineering Avoid AWS Throttling issues Make dyno clients more dynamic and this is a better solution them DNS like route5...

Dynomite is a kick ass cluster/proxy solution that provides high availability and strong consistent to databases. Dynomite was created and battle tested by Netflix using Memcached and Redis as primary store backend. RocksDB it's a Facebook Embedded Key/Value store which is growing up a lot because of his incredible high performance and low latency. For this blog post, I will show some simple project I create for the developers to get started with Dynomite and RocksDB together. This project uses docker and creates a Dynomite cluster using RocksDB as backend. Let's get started...

Cassandra is a kick ass NoSQL Database that guarantees strong consistency.  Inspired by Amazon Dynamo paper Cassandra can deliver high availability and strong consistency.  Battle tested by companies like Apple, eBay, Netflix, Github, Nasa and several others. For this blog post, I will show how to setup a simple Cassandra cluster. This setup is for development purpose and it's not ready for production.

Besides performance, strong consistency and scalability we also pick NoSQL DBs based on the Design of your Data this is one of the cool things about ArangoDB because it has multi-models. ArangoDB is a distributed NoSQL database. It has a flexible data model for documents, graphs, and key-values this is also called multi-model db. ArangoDB has convenient SQL-like query language or JavaScript extensions. In this post i will cover the Document Model and show some have code for such task i will use gradle and Java 8 .  ArangoDB have drivers for lots of languages you check it out here . I'm using ArangoDB in Ubuntu Linux, so to install you just do $ sudo apt-get install arangodb . If you are using another OS or Docker for instance is fine you check it out here as well .

Etcd is a distributed, consistent key-value store for shared configuration and service discovery, with a focus on being. Etcd is very useful in DevOps context because today Immutable Infrastructure is one of the core principles. Often what people do is store the configs into a K/V store like Etcd and them just export OS_VARS into the OS level that could be an Amazon AMI, Linux or Bare metal, Mesos or even a Docker container. One of the cool things about Etcd is that you have a pretty simple REST interface to operate over the K/V store.  When you are doing provisioning sometimes you need store IPs, IDS, Tags, etc... and this is very viable solution.

First all i just want make clear a respect Redis and Salvatore a lot. This is just my view of the topic, no intention to create any kind of FUD or flame war here, just sharing my toughs and experiences. I use redis in production since 2011, its a great piece of technology. Redis has amazing throughput for single instance solution given the fact redis does not have threads(compared with Memcached). Do i recommend use Redis in production? Yes i do :-) Redis: The K/V Solution Whats is Redis? What is good for? Redis is a K/V store and its great for caching. Memcached have few key/set operations in comparison with Redis, there are lots of commands todo very precision operations. Redis cluster was added recently but originally redis was used more like a farm of servers like Memcached case, so replication and other tasks was carry out by the clients apps. Redis has a cluster now. Redis is written in C and has very fast and direct access do data structures like lists, sets and maps. Si...

Memcahed is a K/V store. Its fast pretty much all operations are O(1). Memcached use Threads(Async and non-blocking IO ) different from Redis with is single-thread. Memcached does not have a cluster, its a farm, some concerns like Replication need be handled by the clients or 3rd party application, nodes on the farm don't talk o each other so there is no GOSSIP . Memcached has few key/set operations in comparison with Redis . Redis have several. Some clients have the concept of the master-key / sub-key based on this it can locate what server has that specific key. Often keys are smaller than 250 chars and objects smaller than 1MB. Memcached if used for a lot lot of scenarios like: Profile Caching(Social Networking), Page Caching(HTML), Ad Targeting with cookie / profile tracking, Session caching for games and entertainment its pretty much for caching.

Neo4J is a graph database built in Java with full support for Transactions. It work with nodes and relationships, you're able to put properties in both, graph structure are very simple structures but have great expressiveness and great work for some problems domains. Graph are perfect for social media, because you have friends and the friends have friends and they have twitter, email, cellphone. You do have properties on the relationship like for how long time you know that find or where did you meet him last etc. This is the classical Matrix movie relationship sample. Let's see how we structure this graph on code, I will show ruby code through JRuby. I made some changes and add new properties but the relationship didn't change that much.

NOSQL is a movement is around for long time ago but since 2009 is getting stronger and more consistency. Relation data based have they place and they always will have BUT besides the scalability that in some scenarios is more a matter of cost in other Relations Databases is really the wrong choice not just because scalability BUT is a matter of design and complexity X simplicity. The Right structure have a huge impact on the complexity/simplicity of the code and design and have said that this maybe or may not create performance issues. Is a common sens that Relational Databases have beens overused in this last 3 decades. There a debate about the name, my take on that is that is not only SQL and you must consider different databases or data structures in your solution toolbox.