Diego Pacheco Tech blog

Spring Data Spring Data makes persistence much easier. I personally not a huge fan of JPA and ORMs.  However, I know lots of engineers like it and I respect different preferences. Have different preferences in different services is totally fine. However, doing that in shared libs is NO-NO. Today I want to cover a bit more about how we can use Spring Data JPA and H2 database for testing. Speed matters. Combining H2 and MockMVC we can speed up tests considerably. I made a simple POC showing how we can configure and use Spring Data JPA with H2 and do proper testing using MockMVC capabilities. I also show how to use the H2 Console so we can query the database. Spring-Data makes persistence much more concise and leverages lots of common functionality for us. So Let's get started. 

Spring and Spring Boot provide several ways to test our applications/services. Recently I made a post and video about Testing approaches that might be interesting for you. Previously I also made a post and video about TestRestTemplate  and mocks . For this post, I want to focus a bit on the MockMVC capability. MockMVC has a FakeDispatcher servlet(It will feel pretty real) simulating the MVC stack, however, there are no real network connections being made in practice. So MockMVC provides a lightweight way to test controllers as fast as possible. For this post, we will see a simple POC I wrote with a video going to set up a Spring Boot application and also test some scenarios with the MockMVC feature. So Let's Get Started! 

Threat Analysis is not a new concept. It's being around since the 90s. Threat Modeling is becoming more popular thanks to the DevSecOps movement. There are several other open-source Modeling tools such as  OWASP Threat Dragon and  Microsoft Threat Modeling Tool.  Threagile is an agile and open-source tool for Threat Modeling. The cool thing about Thredagile is that the tool is based on declarative modeling using YAML files. Threagile is built with Go language. One big advantage of open source software is that we can always look at the source code and even change it if we need it(i.e Understand how the RAA score works ). Security awareness is something that still needs to be work with engineering and product teams. This tool is a great way to tear down walls between InfoSec and engineering. Threagile is also dockerized making it very easy to use and play with it. Easily you can version the YAML file in your service repository and generate the reports as part of your J...

This is not a new post, this post is related to other posts and issues that I'm talking about in this blog for at least +10 years. When I started doing SOA I fall in love with it because was all about Services and not about libraries. +15 years doing SOA and this problem still exists in the industry. Dependency management is a lost discipline, and it's super needed nowadays. Every company on earth has internal shared libraries. However, libraries often are a huge trap . There are ways to deal with this issue. The biggest problem often people dont realize is that OSS libs are often maintained by small teams that cannot afford to maintain several old versions of libs for free. Some companies behind popular and big OSS solutions have a paid support service where often you can bigger support for longer. However, that's not the reality for all and the majority of libs out there. Often product teams are focused on release features. Product teams often dont have Slack . The outco...

The first blot post on this series was focus on the data part. This one will be focused on the code and functionality part.  After explaining the Strangler Pattern and understanding some of the options and the challenges, we will cover how we can combine Strangler and CDC as the ultimate solution for Slice and Dice the Monolith. Migrations are hard and often need to be performed into waves. There is a reason for that. Strangler is a code pattern focused on stopping the bleeding and either easily or more painfully migrating to a new codebase from either a legacy or a poorly designed solution. Some people have issues understanding Strangler, but it's an incremental pattern; even when you do things a bit, atomically, it's incremental(minimum 3 steps). 

There are 3 reasons why these patterns might be beneficial and relevant for you. First of all, if you want to replicate a database because you want to run some analytical workload(I made a recent BigData post that might interest you and related to this topic). The second reason could that you want a new cluster or new database technology.  The third use case can be you slice and dice you monolith, and you want to split one service into 2 or even more services. No matter your use cases, the architectural patterns I will describe in this blog post will help you when you need to migrate data at Scale. Dealing with data is not the same as dealing with services. Data migrations are much harder than service migrations. It's pretty easy to do some rolling wave update and migrate service by service and re-deploy even reset them one by one. However, talking about data, there are extra challenges. 

This is not a geography post. However, software architecture and code also suffer from erosion. People come and go, teams get created and destroyed by the project mindset. The result is an ownership problem. Ownership problem is just another form of a Governance issue; Governance is a bad world and often associated with bureaucracy and old ideas, and bad stuff. Ownership issues exist because teams touch a higher number of resources such as Code, Database Schemas, Dashboards, Alerts, Wiki pages, S3 Buckets, Kubernetes clusters, Ec2 instances, and much more assets. So why AWS TAGS are not enough? 

So this is not a post about Slack(IRC/Chat application) - there is nothing about instant group messaging here. Slack also means LOOSE which is the opposite of tight. We all know Debit as bad. We all know Debit slows down execution and increases maintained cost. So why do we still suffer too much about it? IMHO there are several reasons that go from Bad management, Lack of Vision, Poor Tech Talent Pool, Lack of Software Ownership and Culture. Recently I finish Upstream from Dan Heath, which is an amazing book about product discovery but also about Software Architecture and Team Organization(Because both are the same :-) ). Dan has other amazing books like Switch . Which I highly recommend it. 

AI is here to stay. From recommendations to what book read next or what movie or tv show watch next to AI Art generated frames to Self-Driving cars. AI is a powerful technology that we do not master yet. It will bring changes for us as mankind which I'm not sure if we are ready for them. Currently, we have problems understanding WHY ai did something and debug and understand why some outcome happened. This not only is problematic to fight biases but also because we will be doing more complicated things with AI. Today I want to share a simple and short slidecast I made to get you thinking about this. This Slidecast was highly inspired by the books: Life 3.0 and HomoDeus . I also recommend you check out this channel Future of Life Institute and Beneficial AI Panel  and also Sam Harris TED Talk . 

Documentation is always on the extremes. Why? Either a team has no documentation of what so ever or it has documentation paralysis and document every single aspect of the software development process to a point to have waste and slowness and little or no value. IMHO Documentation is part of that group of bad wards like Governance. The reality is Documentation is not what you think it is. There are several ways to document and might not even require to write anything down. Documentation is important. Often people dont know how to deal effectively with documentation and there are so many ways we can extract value from it. So value really depends on the kind of problem we are trying to solve. So let's stop talking about documentation for a moment and let's list some of the problems a software architect needs to help to FIX and then we can revisit documentation later with a freash poblem space lens. Documentation is not about formalizing requirements .

Software architecture it's much more than satisfying requirements. Requirements can be tricky and several times misleading. Requirements are easy answers and they often shut down important discovery processes. Requirements are old as snakes, they were heavily praised by RUP, CMMI, PMI and still in vogue nowadays. The requirements list is just a wish list someone made. Requirements gave the false sensation of doing the right thing but this is just a lie. Requirements are often lying. Software architecture is a continuous process that requires thinking, judgment, leadership, and critique. Requirements are some easy to be gamed and often they are. So why people do it? Well, I think thats the wrong question. Software is full of sudo-science being used for ages and just does not work like estimates for instance. 

Dependencies are natural in big software solutions. However, dealing effectively with dependencies is critical in order to scale and reduce maintenance costs. Team dependencies are not a simple problem so will not be fixed with one simple answer. This is an old problem with most organizations suffers from it either in different degrees or different maturities but the problem is pretty much the same. Architecture is more than technology. Architecture and team organization work closely together and together can provide important mechanisms to scale, maintain, and better deal with team dependencies at Scale.  The Video The Slides Dealing with dependencies de Diego Pacheco Cheers, Diego Pacheco

Dealing with Tests Dependencies is a way to reduce internal complexity and make services easier to maintain and more resilient to migrations and less expensive over time. Several times fixing tests is more expensive than fixing the real code itself. That could happen for several reasons either by lack of trust between teams or because there is a lack of ways to deal with test dependencies issues. Often people don't pay much attention to tests because either is an engineering or QA problem. However, tests are critical for proper CI/CD and at scale require attention. Overtime complexity and tech debt can charge a high price making all future improvements harder, thats why we need to have ways to deal with this problem in the most effective way possible. So Let's get started! 

Yes, Big Data is hard. It's not hard only because of the number of technologies a good data engineering team needs to master technologies such as Spark , Flink , and Kafka Streams (Batch and Streaming), Hadoop , HDFS , and Hive if you have a DW legacy(most likely you do) and the Data Science part of it with Discovery and Execution at Scale. There are needs for different kinds of storage and Design/Modeling, and thats, not even the hard part. The technology landscape gets bigger and bigger as time pass. We have many specializations such as Frontend/Mobile engineering, Backend Engineering, Architecture, DevOps (Which is a movement, not a department, but all companies decide is a role, so you know what I mean), QA(a dying one? ), Product, Management and Data Engineering which often has Data Scientists working with Data Engineers. To some degree, Data Engineering and Data Science have the same issues as Product has today. Unfortunately, the product folks still too much about project m...

Dependency Management is the oldest tricky of the book. DI is useful not only to make it easier to configure a system but also to make it more extensible and testable. Koin is a very lightweight DI framework for Kotlin. Koin is fast, minimalistic, simple, and yet powerful. Today I want to show we can use Koin is a kotlin simple code I build and also how we can do Unit Tests using JUnit 5 and Koin. So Let's get started.

Kotlin has interesting standard APIs and an eco-system. Today I want to cover 3 interesting features in the kotling language: With, TakeIF, and Let. In order to cover that language features on the std SDK, we will end up touching other concepts such as Sring Interpolation / EL and also one of the biggest and interesting language Design choices in Kotlin: The Elvis Operator. I see lots of influence from Scala and Groovy on Kotlin. 

DSL is an expressive way to declare data and even business rules or configurations.DSLs can either be internal or external. Kotlin has interesting support for building internal and external DSLs. 12 days of Kotlin is a great way to learn more about kotlin-idiomatic solutions. Today I will be exploring the 12 days of kotling posts  in regards to DSL we will understand how they work and what options does kotlin provide to build rich DSLs.  So I recorded a video and also coded the sample of the 12 days of kotling post. So Let's get started.  The Video The Code https://github.com/diegopacheco/kotlin-playground/tree/master/kotlin-dsl-fun Cheers, Diego Pacheco

Kotlin is a modern language on the JVM. Created by JetBrains . Kotlin got popular with Android and Gradle however I would say it's an interesting solution also well for backend services. Kotlin has some interesting design choices and handy APIs and a good eco-system, which I will be covering in the next posts. Today I want to show how we can get Kotlin working on IntelliJ IDEA using maven as a build system. Most of the examples you will set out there on the web will be Gradle-based. Don't get me wrong Gradle rocks but maven still the dominant build system in Java land.  So we will see how to configure the build plugins and all dependencies needed in order to make no only Kotlin builds but also integration with Junit and unit tests. So Let's get started. The Video The Code https://github.com/diegopacheco/kotlin-playground/tree/master/kotlin-1.4-fun Cheers, Diego Pacheco

Spring has lots of interesting features to make testing easier. There are many ways we can test our application. Often tests are about confidence but not only that tests are also about having a healthy and clean mindset for maintaining systems. For one reason or another there are migrations that need to happen, either for InfoSec remands or Business wants to have a better set of a new feature for user experience or simply the team wants to make the service right and faster. No matter the reason if you have a healthy set of tests your life will be much easier. The building is easier, maintaining system health requires discipline and tests are a  very important part of it. Today I want to show how easy is for us to write proper and yet lightweight integration tests using Spring Boot 2.x and TestRestTemplate. There is a simple application I wrote and also a video which I will cover the code with more depth. So Let's get started.

Accord is an interesting lib for validations. Accord is composable, simple, self-contained, and integrated with ScalaSpecs for instance. In Java, validations are often done via annotations. In general, there is an abuse of annotations in Java mostly because there is a lack of Macros which we have in Scala and Rust for instance. I really like the Accord approach to validations where we dont throw Exceptions and we can extend and re-use validations across the type system. So today I made a simple code with Scala and Accord using SBT as-built system and a video to explain in more detail. So let's get started. 

I personally have a deep passion for lean libraries. Binary Coupling and distributed monolith kill all services initiatives. Shared libraries are often controlled or done with attention and care. The result is catastrophic, developer experience goes down, the ability to the chance version in isolation goes to space and complexity grows as productivity and migrations become more and more consuming. Tinylog has my respect. I love simple, no 3rd party dependencies libraries. So today I want to show how tinylog is cool and yet powerful. I made a video and a simple Scala project using SBT. So Let's get started, 

Memcached is a powerful and battle-tested key-value store. Netflix is one of the many companies that use Memcached at scale. Memcached has a simple design and yet powerful. Redis is great because of the commands and data structure diversity however Redis is not multi-thread and Memcached is. Thats the biggest advantage of Memcached. Scala, it's a powerful and complete modern post-functional language for the JVM. Today I want to show how we can use MemcacheD and Scala together using the Shade library. Shade is part of monix which is a functional multi-thread solution. Today I made a video showing how we can use Memcached and shade in Scala using SBT as a build system. So without further due, let's get started. 

Rust it's a very nice and powerful language. I believe Rust has a place beyond systems and hardware programming. I personally believe we will be seeing more and more Rust in the following years just like Go took some time to liftoff I believe Rust time is yet to come. Rust it's powerful because favors safe programming not only in sense of memory management but also combined with functional programming we can get the best of all words meaning: Safety, Productivity, Less code, Doing more with less, reducing complexity, and leverage engineering best practices. Today I want to show how easily we can write a cyclic state machine parser with Rust and without using any crates but really 100% on the standard language.  So Let's get started.

Security it's super important. Unfortunately, security is often forgotten and the price comes at scale and harassment to your brand and user experience. Spring-Boot is super popular among java solutions in the tech industry at the same time people often run servers without proper TLS configured. There is much mysticism and wrong vision about security. Secure Spring Boot using Tomcat for instance is not hard at all.  Today I want to share a video and code I made showing how we can secure the Spring-Boot 2.x application using Tomcat 9 and doing redirects where users try to reach 8080 port. Customers facing apps should definitely use TLS and HTTPS. However, it's so easy to do for the backend and the overhead is not that bad that we should do it as well. So let's get started. 

Spring Boot has interesting extensions for mocks using Mockito. Sometimes mocks can be very harried. Unit Tests make sense for services however the cost of mocks sometimes is too high and it could be a very time-consuming and complicated task. Spring-Boot has interesting support together with mockito to make mocks easier and more declarative. Mock is not a free lunch dont get me wrong. However, when you are coding a service Dependency Injection makes sense. Since Dependency Injection makes sense for services why such a thing would not make sense for mocks? Well, it's the same principle. If your implementation relies on dependency injection, your tests should also do the same. I have a bit of different opinion about shared libraries where I dont think they should be coupled with Spring and therefore might not have DI at All. Libraries and Services are completely different animals. Going back to services, today I want to share a video and some sample code I made to show how mocking c...