Posts

Showing posts from 2024

State

Image
If you look up on dictionary.com the first two definitions of state are: the condition of a person or thing, as with respect to circumstances or attributes: a state of health. The condition of matter with respect to structure, form, constitution, phase, or the like: water in a gaseous state. Software is all about data, which is the state. The state could be your basic data, such as your name, last name, address, phone, email, bank account ID, or any other IDs and information you might have. Buying software has many side effects, many of which  are negative . Such data needs to be captured somewhere, usually in a relational database. But it could be in any other data store like a NoSQL Database like Cassandra, a K/V store like Redis, or an object store like S3. Ideally, the state should be contained and even isolated. However, in distributed systems, we often need a distributed state. The state is necessary. However, we might create additional complexity if we don't deal with the s...

Leaky Contracts

Image
Service contract design is hard. People do it all the time, but it is not always correct. Most of the time, engineers pay more attention to the service's internal implementation than the contract. A Service contract is the most important part of a service. A good contract provides an abstraction that allows us to have flexibility and allow future refactoring without impacting clients. Being able to refactor code without breaking consumers is pure gold. It's priceless. Not all industries are the same; however, the more dynamic and agile the industry is, the more you need to be able to do Refactoring. To do refactoring more often, you need to make decisions that allow refactoring to happen at a lower cost. If the price of refactoring is always high, no refactoring will happen, and that is game over; that is how we create inadequate legacy systems. A service contract provides a high level of abstraction(ideally) or might(if they do a poor job) leak internal implementation details....

Services

Image
We are in the holiday season. You walk into any Starbucks and see the Christmas decorations. Lights and Christmas trees are everywhere, and the advent of code is still rolling. Whether Xmas is summer or winter, You can get some specials and flavors only on Xmas. Now, you must wonder if this is a technical blog post; yes, 100% it is. I promise there will be a point at the end. Let's imagine a very different coffee shop experience; in reality, it's not how I will describe it, but hopefully, it will help you to see my point. I'm pretty sure you've gone to a coffee shop before; how it works is simple: you look many for what coffee + snacks you want, you get in line, pay at the cashier, and then the barista will brew you coffee, warm your food, and you will be good to go. If it is a good place, they will call you by name. Now, let's imagine a very different experience. Consider this the upside-down Xmas experience. However, if you want to name it, I will give you my nam...

Proprietary Systems and Distributed Monoliths

Image
Distributed Monoliths are the predominant form of modern legacy systems. Sometimes distributed monoliths are created by teams misapplying microservices architecture due to lack of isolation or internally shared libraries break of isolation. Other times, there is just the proverbial no time to refactor. Let's share the database. However, sometimes distributed monoliths happen due to non-obvious reasons from decisions that, at first glance, sound like good ideas and are even far away from engineering.  In software, we often run a trade-off analysis, Build vs Buy, where we must decide whether to build or buy software. Buying is usually attractive because it gives some speed (since you don't need to code it). Building, of course, takes longer, and you will need to maintain it; there is a chance of the project failing, and you invest money and have no software. There are trade-offs between building and buying. However, people often think buying has no implications and is much better...

Expectations

Image
Every single day, engineering teams need to do magic. Such magic manifests in several different shapes and forms. It could be by making hundreds of assumptions, figuring out what to do with little to zero context, and delivering decent solutions with proper architecture, design, comprehensive testing, and value via continuous delivery. It's a lot. In reality, most industries use Scrum/SAFE, which is far from the best way to work. In theory, when you are doing Scrum, if you do not finish a story during the spring, it's not a big deal; you go to the next one, shift the stories to the following spring, and it's all good. However, in reality, it is not exactly like that, unfortunately. That happens for various reasons: ignorance of what proper agile looks like, irrational pressure, industry pressure, and hundreds of other uncontrolled and hidden variables. Lean is all about learning to see waste, remove it, optimize it, and improve it with more quality. Companies, unfortunately...

Functional Programming

Image
There are many programming languages. Most of them are based on C. However, there are a few different schools of thought. There are imperative, object-oriented, and functional schools and a couple of branches and variations, but that's the mainstream of languages you will see in production in big companies. Programming languages is how we express our intents and turn that intent into value for people. Usually, people are concerned about how other engineers would read the code, make sense of it, and be able to effectively maintain it. System programming languages like C and Zig tend to focus more on performance over abstractions (which always have a cost). Object-oriented programming languages like Java focus more on structure and how to abstract things. Abstractions are not necessarily bad if done right. They have a cost and impact on the people who maintain them. Having more abstractions on the SDK or standard library is a good thing for mantainance and increases people's kno...

My third book is out: Continuous Modernization

Image
After 7+ months of hard work, my third book is out. Introducing:  Continuous Modernization : The never-ending discipline of improving microservices, monoliths, distributed monoliths, individuals, and teams at scale. Modernization is something I have done over and over in my life. Modernization is something that all companies need and will always need. The need for modernization will never go away, even with LLMs and AI. Technical debt, Anti-patterns, and bad decisions do not take days off. Complexity never shirks and continually grows. Companies do not stop getting bigger and doing more and more software critical to growth, delivering value to customers, and staying competitive and relevant in the market. We need a better way of doing software to drive the best outcomes out of us. Continuous modernization is the counterforce to technical debt and anti-patterns. Continuous modernization is how to continually improve, even when you think it's impossible and nothing can be done. We ca...

Proper Error Handling

Image
No matter what programming languages you use. Engineers need to make dozens to hundreds of small decisions every day. Such decisions can sometimes save us and, other times, create many problems. Some of these decisions can be called assumptions. Depending on the context, they could be the side-effect, lack of proper discovery, feature factory rushing to delivery, or lack of care. In reality, error handling is one of the most challenging things in computer science, alongside naming, cache invalidation, and off-by-one errors. We usually get error handling wrong. When we are supposed to throw an error/exception and crash the application, we are unlikely to return some bad, sneaky default that will produce a bug down the road; when we need to ignore missing information, we end up crashing the app. Code reviews rarely to never review error handling. It's common to see services in production that don't even have the proper exception track due to improper error logging. Error handling...

Legacy Systems and Distributed Monoliths

Image
We can't have all the software in one system, even if we try. By nature, distribution will always happen. Decades ago, SOA set the standard for creating proper services that prime interoperability. Distribution is not bad; it's how systems work at scale. When companies buy products, they add more distribution because there is no system in a complete vacuum. Now, when we have distribution instead of modules in a monolith, we need to have a way for the system to communicate. When we have a proprietary system, communication options can often be pretty limited. Ideally, communication is thought ahead of time in an approach we call contract first. Where we define contracts and explicitly manage contracts aiming for interoperability. Usually done via a common interoperable interface like HTTP or, nowadays HTTP/2 or even HTTP/3. Legacy systems usually are not API friendly and, therefore, limit your ways of comunication. Such limitations can be drastic to the point where you might only...

The Dark Side of LLMs

Image
AI is the most considerable hype right now.  It is not as new as people think, starting in the 1950s. Significant advances have happened since 2017 with the  transformers architecture , which is the heart of all generative AI. There is an excellent potential for substantial disruption  because of AI. We are seeing significant improvements, but we are far from AGI. AI can be practical and honest, add value to the business, and improve our lives. AI is narrow at the moment and has many challenges. One of the industries that has the potential to be highly disrupted by AI is the technology industries and engineers.  Large Language Models (LLM) can do amazing things. From generating text, generating creative images and videos(with lots of problems), and even generating code. It's absolutely normal to be concerned, but the more you understand what's actually going on, the less you need to be worried. If you are an expert, you will be fine. We saw a great leap and boost of...

Testing Queues and Batch Jobs

Image
Testing could be considered a solved problem. Everybody knows the importance of testing. Unit testing and integration testing are not rocket science. However, it is still not uncommon to see a lack of testing, poor coverage, and flaky tests in the industry. Internal service implementation can be tested using fakes and mocks. Testing classes is a simple task if you have a good design. Refactoring a legacy system to have a better design makes it easier to test and could be more entangled, but it is still possible and desired. However, sometimes you have a good design, which is still hard to test. Some architectures can be more complex to test naturally. If we consider standard RPC services, it's pretty vanilla to test them. However, things get more messy when we consider integration tests or end-to-end testing, mainly because of dependencies and state. From the test point of view, RPC services are simple. You have a request, very likely a rest call, and you have some assumptions; you...

Frontend / Backend Distributed Monolith

Image
The current trend in front-end development is to be full stack. Being full-stack, has advantages from an efficiency point of view. However, having people proficient in both the front and backend can be challenging. The trend is that many javascript frameworks, initially only client-side, are moving to the backend and becoming full-stack frameworks. Backend change is happening gradually, probably starting with server-side rendering ( SSR ). Now, it is possible to have react code in the backend, which is referred to as react server components ( RSC ). Such an approach is not new. PHP , Rails , and Django have been doing that for decades now. Within similar timing, we saw the rise of HTMX where the backend levering HATEOAS returns HTML rather than just returning JSON. Interesting times. React is not the only framework (because it's not a lib) doing such a thing; we also have  Next.JS (on top of React), Svelte , Nuxt  and here is a good article on server components if you wan...