Tim Bannister
2 March, 2022

The promise of cloud-native is frequent, worthwhile improvements without undue effort.

My previous article quoted Alfred North Whitehead, who said that “civilization advances by extending the number of important operations which we can perform without thinking about them”.

I’m going to propose a kind of corollary: compromises occur by extending the number of operations we perform without thinking them through. Although I’m sure some folks would suggest the answer is thorough scrutiny for even the smallest change, most sectors don’t need that extra effort.

For some common cases, here are my recommendations on what tech to use and why.

Tech choices

APIs

In the case of API keys, the main factors that influence design choices are lifespan and risk. Use a managed service such as AWS Secrets Manager even for low-risk API credentials. For more sensitive secrets, use partitioning if feasible: protect a long-lived token in a managed service, or use inherent resource credentials to manage access. Then exchange the long-lived token for a time- and scope- bound access key that you pass to application code.

A trickier choice is when you’re the client, accessing a critical service, and the provider only offers a single privileged API key. Automatic key rotation helps manage risks, but a compromised component in your code still has full access. Instead, another partitioning approach works. Implement a trusted proxy with its own access control; use your cloud provider’s own service if you can. Store the privileged API key in a managed component that enforces access controls (such as AWS Secrets Manager), and grant read access to the proxy. You don’t even need to let the key rotation tool have read access to the key - that component can go through the proxy and have its invocations logged, just like any other component.

To protect data in transit, use TLS (and ideally, enforce that the TLS version is at least v1.2, as earlier versions have some important security shortcomings). Use a managed cloud load balancer unless your specific case justifies additional measures.

You could add client-side encryption: have the application, such as JavaScript code in the web app, invoke your component with a payload that is already encrypted. You can provide a public key to the client and verify on receipt that the message matches a known private key. You can do all that without actually decrypting it, too (sometimes you want to leave that decryption to a specific microservice).

This is a strong way to minimise the impact of a breach: if the APIs you run aren’t allowed to decrypt the sensitive data, an attack becomes more difficult, and an accidental leak less serious. You’re additional protected against some breaches of infrastructure elements such as load balancers and CDNs.

Customer data

For documents, store those in object storage using managed encryption if available (on Amazon, use S3 KMS, and not the simpler AES256 mechanism). Set controls on the key you use so that both object reads and object decryption are restricted. If appropriate, spend the time to implement key replacement, including steps to re-encrypt existing records under the new key.

It’s also OK to use someone else’s API for documents, so long as you use that securely.

Protect sensitive queryable data using managed relational databases such as Postgresql that provide row-level security, column-level access controls, and usually also field-level encryption, on top of a foundational layer of managed encryption at rest.

Whilst alternative technologies such as graph and document databases offer some options in this area, relational databases seem to offer the strongest and most effective mechanisms.

With an RDBMS, access controls for rows and columns work entirely within the database, whilst for field level encryption you usually need to involve keys and key management. For effective implementation, rely on a separate (and managed) service to hold those encryption keys. If the keys are part of the database, are they genuinely providing the isolation you expect?

Here’s one place you need to understand the risks for different data paths. If a table works more like a mailbox, you can use column-level access. For defence in depth, add asymmetric encryption and implement separate paths for reads versus writes. For the mailbox table example, set things up so that the app needs to call a microservice to get back the plaintext.

For a different example, apps often need to query against PII. Picture a database holding sensitive personal records. You can use tokenisation so that instead of placing name or address in the row for a person, a foreign key relationship points to a different table where the actual values are held, hashed or encrypted. These additional controls mean that even a full database dump does not directly reveal sensitive information.

Consider a partitioning approach if you need to query some fields but keep others confidential. Take a personnel records system as an example, where you have a feature to hold staff photos and also let colleagues verify identity. Fields like name or employee number would be indexed for searching, so locked-down encryption isn’t helpful there.

Because your code never needs to query the photos, you design a partitioned application where the client part of the app fetches a key and encrypts the picture before uploading it. To decrypt the photo, the app client can request decryption.

Containers and secrets

The story with ECS is easier to secure than EKS (AWS’ managed container platform).

You can define an ECS task definition that looks up a secret from a source (either Secrets Manager or SSM Parameter Store) that you specify by ARN. Provided that you grant ECS access to read that secret, it all works how you want.

For example, here’s a snippet from a task definition.

    "environment": [{
        "name": "examplePublicValue",
        "value": "https://this-is-not-confidential.example/"
    }],
    "secrets": [{
      "name": "exampleConfidentialValue",
      "valueFrom": "arn:aws:secretsmanager:eu-west-2:012345789012:secret:b3BlbnNlc2FtZQ"
    }]

The confidential option exampleConfidentialValue is loaded at task startup time so it’s not part of the task definition.

Using Secrets Manager is a better fit than SSM Parameter Store, because Secrets Manager data is always encrypted, and because Secrets Manager supports resource-level access policies.

If you’re using Kubernetes, protecting Secrets at rest is possible too - but it’s nothing like as straightforward. You get basic protection from using the Secret API, but beware that out-of-the-box Kubernetes doesn’t have any resource-level controls on access.

That advice for ECS is sound for all but the most critical workloads, but for Kubernetes I’m afraid that there isn’t a similarly straightforward tip that gets you strong security. Using a Secret helps but there certainly are details there that you, the AWS customer, are expected to manage - and there’s no one-size-fits-all means to mitigate those.

Generic encryption

You can pay to have a managed service help with your encryption needs, backed by hardware security modules.

On AWS, that means KMS (there is a lower-level service, CloudHSM, that leaves you lots of work to implement yourself). AWS KMS lets you generate a master key within the service, and then use mechanisms such as envelope encryption to protect different records. Controls on key use let you make sure that users can only decrypt data they are authorised to view.

You can use asymmetric encryption in two different ways: the first based on maths and cryptography, where there’s a public key and a separate private key that’s stored within the hardware security module(s). The second way is with symmetric encryption and access policies, so that a wide set of principals can encrypt data but only trusted principals are allowed to decrypt it.

If you need to roll your own thing, leaving as much of it as possible to be managed by your cloud provider is a sound approach to follow.

Overall

There are lots more ways to manage privacy and confidentiality in the cloud. If I could pick only one message to emphasise, it would be that you should prefer to use managed cloud services. Building something yourself with similar security guarantees is really a lot of work.

Across the different technologies I’ve touched on, there is a theme, too: keeping things secret in the cloud, access controls play at least a strong a role as the encryption technology.

Designing effective systems security for your SaaS business can feel like a distraction from delivering customer value. Book a security review today.

This blog is written exclusively by The Scale Factory team. We do not accept external contributions.