Why Encryption is so hard?
Basic Concepts
Basic Security means having data protected at Transit which means TLS/mTLS modern Service meshes like Istio provide that out of the box. However, that is not enough, we also need to secure Data at REST which means on the database storage layer. That can be done by a variety of methods like:
* File-System Encryption: You secure the underlying filesystem(EFS) - This solution is often simple to implement and transparent for the application.
* Database Encryption: This is an interesting approach where your datastore encrypts the data for you which is great for the application in sense of transparency however it could lead to performance bottlenecks depending on the implementation.
* Application Level Encryption: This is the most flexible way of encryption however it has a price, you might end up cripptle databases capabilities. For instance, relational databases might be lost Query capabilities such as Like. Non-relational databases like Redis might also lose lots of functionalities and others such as ElasticSearch might be very hard to use with App-level encryption.
Encryption Apis are Hard
Encryption APIs are often hard to use. Not only the design tends to be complex and obscure but also important parameters for security such as how to initialize a secure random key and handling IVs properly are often not well documented.
According to the paper: Java Cryptographic Uses in the wild:
* 85% of cryptographic APIs are misused.
* 72% of Java projects on Github have at least 1 security misusage.
* Most common Java JCA Apis such as MessageDigest has 92% misusage.
Encryption Apis are hard to understand and have a poor design which leads to poor security. Security is also a design problem. How can you design a good API and make usage easier if you dont understand what you are doing it?
Challenges with Symmetrical Encryption
So encryption APIs are poorly designed and hard to use but besides that is all good right? Well not really, there are so many other problems related to Encryption. First of All, you could reach performance and Scalability issues. Encryption, especially at Scale can be slow and depending on your access patterns and volume of data. In order to deal with these problems, you might need to apply software architecture and design techniques such as Batching, Caching, and Partitioning.
Another problem symmetrical encryption faces is the fact that the Product Organizations often prioritize features instead of Technical Debt, Tests, and Security. Add the fact that loopholes, breaches, and vulnerabilities are found daily and we need constantly update software. Security is a never-ending battle.
Encryption is about Keys and it's about permissions as well. Permission is also a hard problem. Especially if you are not running your workloads on the cloud and cannot really on IAM permissions. How do you know what applications need to access what keys? Distributed Monoliths could also impose challenges for security and Isolated Proper SOA Services which does not share keys can be a big relief.
Data Breaches happens, unfortunately. For that case, you need to be able to detect what keys leaked and Rotate the keys. However, besides Data breaches is often a good idea to rotate keys periodically no matter what. Data Rotations can be a challenge at scale since you could be disrupting the user experience while rotating the keys and ciphertext.
It's all about Migrations
Technical debt often makes security harder. Since attackers never stop you are in a never-ending battle and no matter if you release new features or not, you might need to perform security hygiene tasks often. Cloud computing makes everything easier but at the same time, it demands more security. Amazon Web Services CTO Werner Vogels said in 2018 that Security is everybody's job. That's true no matter if you have an infosec department or not. Movements like DevSecOps gain strength over the years.
The big issue is if you have hundreds or even tousands of microservices you need to run constant migrations in order to fix security gaps or even your own encryption mistakes. There are architecture techniques that can help you such as Side Cars and Service Meshes. However, even if you are doubling down into Service there are still reasons that could lead you to full and constant migration such as Contract Change due to new requirements. Libraries can also be used but you need to understand that there is no free lunch and you could be falling under other forms of traps.
The technology industry is not even close to being done in a sense of security. 10 years ago was common to pitch engineers the need for unit tests and writing tests also an important role in software engineering not only a QA job. Now we still find companies with a lack of tests and issues to introduce CI/CD but not only that need to be fixed but we also need to pick engineers to care and understand more about security and encryption.
A big challenge in regards to encryption and security, in general, is that is not a one-side issue(let's say infosec, secops, or security department). It requires engineering to partner and works together in pragmatical and secure solutions, which requires balance and close partnership. Digital products that do not operate under-regulated industries need to care about security and encryption otherwise a data breach could destroy your brand and harass your customer's trust.
The DevOps Movement was marked by the wall dividing Dev and Ops. DevSecOps is the same story as InfoSec/Sec and DevOps. We need to tear down the wall and make security learn more about engineering and engineering to learn more about security.
Cheers,
Diego Pacheco
