Disclaimer: This post was published a while ago and may be out of date
I am a big fan of speed limits. Every time I cross a road, or open my flat window at night, I appreciate that everyone has collectively agreed that we don’t drive our cars faster than 30 miles per hour in a residential area. The speed limits are there to protect everyone, the driver, other road users and pedestrians all. AWS service quotas (formerly known as limits) are the speed limits of the cloud. They are there to protect you from unexpectedly large bills. For AWS, they help them protect service endpoints and manage utilization across tenants.
But at the end of the day, service quotas are limits. Limits that may restrict your SaaS business’s ability to achieve its goals. Thankfully, just like we as a society have all agreed that ambulances and F1 racetracks don’t follow the usual restrictions on speed, AWS have agreed that there are circumstances when service quotas should be configurable. The trick is getting them increased before they start limiting your ability to execute.
Why Should You Care About AWS Service Quotas?
If you’re running SaaS workloads on AWS, it’s highly likely that you’ve already requested service quota increases on your main workload accounts. If not, you probably will as you scale up. For example, if you run multiple tenants per account, you may be worried about a single tenant using up your concurrent lambda executions quota and affecting your other customers’ performance. Lambda lets you reserve concurrency to protect other tenants from a noisy neighbour. To make that possible, you must have an account-level service quota that’s at least as high as all the reserved concurrency settings added together.
Even if isolating tenants at the account level, you could find larger customers hitting quotas all by themselves. In these cases, you may wish to make increase these accounts’ quotas independently to keep the safeguards for your smaller tenants.
Some AWS quotas are surprisingly low at their default values, and your application’s architecture may just require more than the default quota at a baseline level. Whatever the reasoning, when you hit these limits there will immediately be business impact, be it service degradation or limiting your ability to onboard new clients.
Always keep in mind that some service quotas can only be increased up to a maximum level, and some can’t be increased at all. For these quotas, you may want to consider sharding across multiple regions or accounts, or otherwise re-architecting your solution to reduce your consumption.
When Should You Request Increases to Minimise Impact on Your SaaS Offering?
The best time to request a quota increase is ahead of time. First, you’ll want to identify which quotas may need to be increased in the future. For day to day operations, this can be done with some well crafted alerting rules in your observability platform of choice. Set alerts for when your actual usage is approaching your quota to give you some time to request increases before hitting that wall.
For disaster recovery or scaling scenarios where you may be starting fresh in a new account or region, it can be a little trickier. A good way to bridge this gap is to rehearse your disaster recovery plan with fresh regions or accounts. You may find yourself immediately hitting your quotas and should then build those quota increases into your runbooks. Doing some load testing on your simulated recovery environment can also help you identify limits that may only present themselves once you’ve scaled to production levels.
How Do You Request Quota Increases for Your AWS Accounts?
It used to be that every time you wanted to increase your service limits you had to have a conversation with AWS support. Thankfully, there is now a service for that, appropriately named “Service Quotas”. Generally speaking, the process is now to search for the quota you wish to increase, enter the new value, and then send off your request. Sometimes these increases may even be automatically applied. As this is a service with an API, you can even use infrastructure as code tools to make the requests for you.
However, when asking for a quota increase then you need to keep two things in mind:
- Quota increases are not instant.
For many quota increases, there may be a lag between approval and seeing changes in your account. It can take up to an hour for some increases to come into effect, time which you should bake into your DR plans.
- You might get pushback.
With quota increases that don’t get automatically approved, you might get some questions from Amazon support. They’ll ask you to justify the quota increase and approve based on whether they think your use case is valid. Remember, the quotas are there to protect Amazon as well as you. You’ll be best served by providing a brief description of your architecture and the business case it is solving. For DR scenarios, it’s best to have a menu of likely needs and justifications ready to go so that the team can share these with AWS Support as needed. If you are struggling to get an increase through, you may want to ask your friendly AWS account manager for some help or advice.
Another option is to get quota increases requested when provisioning new accounts. Quota request templates can be configured in your AWS organization’s management account, these templates allow you to automatically ask for up to 10 quota increases on your new accounts as they are initially provisioned. In combination with Control Tower this can greatly simplify your SaaS account provisioning story.
Don’t let cloud service quotas kill your SaaS. Make sure you understand what quotas are relevant to your business and if possible raise them ahead of time. When planning your disaster recovery processes, make sure quota increases aren’t going to slow down your recovery times.
This blog is written exclusively by The Scale Factory team. We do not accept external contributions.