Dependency Management Best Practices

Every single technology project uses dependencies. No matter the language you use, you always will use dependencies. Monoliths and Monorepos make dependency management more manageable by having it all in the same place. Services and Microservices make them a bit more challenging because now they are spread across each server, updating libs is a challenge that requires automation to be fixed. Besides automation, some best practices are also needed. Several times engineering teams take dependency management for granted. After all, we are just adding some strings and numbers to a file, right? So how complicated can that get? Often dependency management issues only appear after time and scale. Dependency Management is a super important and relevant discipline. Today I want to share some best practices to make your life better and make sure you scale your codebase with speed and solid practices rather than piles of tech debt and pain. So In case your not geek enough, the post icon image is a famous Death Star for Star Wars, which also becomes popular by the Death Star diagram from microservice organizations like Amazon, Netflix, Twitter, and other companies. 

Dependency Management Best Practices

Often Best practices are depending on contexts. Once you have complexity several times, best practices do not necessarily translate from one company to another; however, there are cases where they apply and make sense, not always, but for Dependency management IMHO, these are Golden Rules and excellent practices to be followed. Here are some Dependency Management best practices: 

• 1. Use a Dependency management tool (ant, maven, gradle). Do Explicit Dependency management.
• 2. Use Artifact Management Solution (Nexus, Archiva, Artifactory) - Management but Mainly: Cache.
• 3. Remove dependencies you dont use.
• 4. Use Consistency Versioning (MAJOR, MINOR, SEC/PATCH).
• 5. Do not use multi-project EXTERNAL POMS.
• 6. Keep Dependencies up to Date (But does not update at deploy time - Immutable Infrastructure)
• 7. Use Dependencies Carefully (Shared-Libs) avoid coupling as much as you can.

Dependency management is not only about using some specific or better tools. It's about a process and culture which requires attention and automation as well.  Let's do a deep dive into each of these practices and understand why they are important. 

Use the Dependency management tool (ant, maven, gradle). Do Explicit Dependency management.

It might sound obvious but it's not uncommon to see infrastructure projects downloading binaries manually and not using explicit dependency management via tools like Ant/ivy, Maven, and Gradle. No Matter the language you use, no matter if is an engineering or DevOps code, you should do explicit dependency management. Because is easier to maintain and we can relly on common process and tools for improvements and housekeeping. 

Explicit dependency management means, explicit defining dependencies on a file that is used by the dependency management tool. You should avoid having embedded dependency and scripts who download dependencies outside of your main tool. 

Use Artifact Management Solution (Nexus, Archiva, Artifactory) - Management but Mainly: Cache

Software tends to grow with your business. As you grew build times can get very slow. A cache is a must-have feature. Because there are multiple engineers downloading artifacts from the web and you often have multiple cloud environments like DEV, STAGING, STREE, PROD, etc... Dependency management solutions like Nexus, Archiva, Artifactory, also can help with better dependency management but one of the main benefits is to have a central cache and central repository management/location. 

Remove dependencies you dont use.

This might sound silly. But un-used dependencies are bad as Dead Code because they make upgrade efforts harder and they end up creating technical debt. It's not uncommon that dependencies have 3rd party dependencies too and you might be dependency from a 3rd party dep instead from a direct dep and have that scenario for a dep you dont use is bad. It's unclear, confusing, raise false positives, and makes reasoning about refactoring efforts much much harder. 

Use Consistency Versioning (MAJOR, MINOR, SEC/PATCH).

Versioning is something old as the snakes in the jungle(like we use to say in Brazil). However people still dont get it right. Why? People know when to use MAJOR(Major API Breaking change), Minor(Minor change no breaking backward compatibility), and Security/Patch release (minor bug fixe or security patch, not impact). But people do not do it. Why? Most of the time is a combination of lack of discipline and lack of ownership and pain of upgrading people dependencies. Central teams can be great in sense of reducing some costs but certainly, they hide some of the pains that if people would face them directly the would definitely deal with the problem differently.  Having consistent versioning is super important, for Design, for Testing, and for health engineering practice I would argue. This part requires discipline and every single binary should embrace this principle. 

Do not use multi-project EXTERNAL POMS.

Don't be fooled by the word POM. This principle works for any dependency management tool. You should not share multi-project external configs for dependency management. Either you have a monolith or monorepo where you have all code in one place or if you do have multiple github repositories you should not share these files(poms). Because? Well because they are EVIL. They create coupling they make upgrades harder and they kill microservices. 

If you will have shared libraries they should be:

 * Small

 * Independent 

 * Isolated (dont have poms, not share configs)

Otherwise, you will build a distributed monolith and binary coupling will prevent you from upgrade when you need it. Never trade coupling for convenience or developer experience.  However, if you have a monolith or a monorepo is perfectly fine to share poms. 

Keep Dependencies up to Date (But does not update at deploy time - Immutable Infrastructure)

Another super important practice is to keep your dependencies updates. Thats important for several reasons such as:

  * Prevent Bugs

  * Fix Security Bugs

  * Reduce Tech Debt

Update Dependencies often works with the same principles as Branches in Configuration Management. If you gonna have a long-lived branch(which you should avoid at all costs) you need to do merges every day so it reduces the complexity and issues on an old fashion bing bang boom merge. Libraries updates work in the same way. For minor, security patches, even minors should be able to upgrade it easily. 

DevOps has a principle called - Immutable Infrastructure, you do not want to upgrade libs before doing a deploys or when a service restart. Because that breaks the principle of immutable infrastructure. However, at the same time, you want to AUTOMATE your dependency management and update libs frequently. Often engineers do not have the mindset to keep updating libs, which can be fixed with proper plugins and automation. 

Use Dependencies Carefully (Shared-Libs) avoid coupling as much as you can.

When we ship internal shared libraries we need to be very careful. Shared Libs should be treated by applying the same principles we apply for Services. It's super important to pay extra attention to 3rd party deps in shared libs in order to avoid binary coupling. It's fine to use shared-libs, sometimes thats the right solution, however, there is a huge abuse of internal shared libs on the technology industry. 

Better Dependency management helps Design and Testing. It makes CI/CD more effective and in the long-run increases the speed and ability to ship better and more frequent software. Currently, we live in an era where every company is trying to do proper CI/CD, Observability, Services, DevOps, SRE, and many other important matters however we often forget dependency management is an important sub-part of Building that end ups charging a high price at scale. 

Cheers,

Diego Pacheco