Languages over Libraries
Libraries provide functionality, code reuse and allow us to be productive to some degree. I’m not against the usage of libraries. However, we need to understand that libraries have issues and hidden problems. For instance, productivity cannot be measured only on how fast you get things done in the sense of days, number of user stories, or releases. Several side effects need to be taken into account such as:
Bugs: How many bugs did you produce by direct or indirect usage of a library.
Incidents: If the bug creates a major incident, how much money did you lose, and how much your customers got affected? Any brand damage or loss of trust of your users?
Debugging & Troubleshooting: How many hours, do teams spend debugging code and making sense of complex problems in non-obvious code branches and scenarios?
Tech Debt Gravity: Technical debt does not get fixed by itself. For sure the best way to pay technical debt is by decommissioning software. If an internal shared library is used across hundreds of services, can you easily kill it and decommission the library? The answer is no. Tech debt has gravity, and it is not easy to bypass the inertia. It is possible but not easy.
Documentation & Hiring: Can you look on the internet? Internal shared libraries do not have content on the internet. You cannot google, look at stack overflow. You cannot hire people with experience if it is not open source.
Entropy: Whatever the library is doing for you, you still need to integrate with the rest of your ecosystem and your other systems and APIs. Even using a library does not make your job 100% done. If the library is not maintained, entropy will kick in, migrations will be more difficult, and security patches will be more difficult.
Internal Shared Libraries have many other issues. As you can see, there are drawbacks and such drawbacks need to be evaluated in proper tradeoff analysis. What often happens is the opposite. Engineers do not perform tradeoff analysis, often google for libraries that do what they want and add to the project.
Mastering your Tools
Seasoned engineers know their tools, by spending more time reading the standard library, looking for existing library documentation, and even leveraging simple utilities in Linux, python, and the JDK.
Several times you might have a function or method that can help you. Other times it is a matter of thinking a little bit differently. We can reverse the flow or thinking. Problems can be fixed in a variety of ways.
Languages are powerful
Programming languages have amazing properties. By definition, languages are much more powerful than internal shared libraries. Languages shine because:
Allow us to think differently: Languages allow us to express different ideas, concepts, and different ways to solve problems. For instance, Java is good for the backend, Typescript is good for the frontend, Zig and Rust and good for embedded systems.
Languages Gravity: Languages have stronger gravity than libraries. People join or quit companies because they want to work with some languages or they are sick of working with other languages. In other words, languages attract or repel talent. If your company is strict and very limited about what languages you can use, you will limit what kind of talent you will get. Languages are important for hiring and retention.
Languages Have Better Support: Compare with internal shared libraries and often more up-to-date with vulnerabilities and patches. Languages are often more backward compatible than libraries.
Languages Can be Transparent: When you add more languages you are not limiting your options, you are opening your options. If you practice proper SOA with isolation, it does not matter what language you use, your consumers will not see nor care.
Documentation: Languages will have orders of magnitude more documentation than your internal shared library. There will be conferences, videos, stack overflow, and people with experience that you can hire.
Simply put, when libraries are awesome, companies open-source such libraries. However, when Internal shared libraries suck they never get to see the sun outside your company. It is the perfect trap waiting for new engineers to fall into them over, and over and over.
Internal Shared Libraries is one of the biggest fallacies in modern software engineering. Why do you need to create more than 10 libraries? Why are the ones that exist on the internet, not good enough? Often people say stuff like “We are Special“ for many years, decades ago, pretty common at the beginning of the agile movement. Other comments like “X does not Scale,” and we need to make it scale. Other comments like “Y does not do Z” are all examples of famous last words.
While the more internal you are, the less open source you use, the harder the onboarding for engineers will be. Over time software tends to decay, using more open source is a good alternative to create lots of internal shared libraries. Companies try to justify internal library creation for all the wrong reasons like lack of trust in engineers, fear of bugs, and even job security.
Seasoned engineers program in multiple languages. Will use the right tool for the job. When people argue that it is better to normalize in one language for reuse, that can be true to some degree. Especially for tooling for CI/CD, observability, etc… However decent solutions have SDKs for all popular and mainstream languages, like AWS SDKs for instance.
More languages are the way to go. More Languages over libraries, meaning go use more open source, build much fewer internal shared libraries, and try to count internal shared libraries with one or two hands max.
