The thoughts throughout this guide are focused on EC2 instances unless otherwise noted.
AWS offers a number of EC2 instance classifications, here we want to make sure we’re considering the type of workload on the instance (is it CPU or memory bound), our networking and storage requirements and if we’re configuring our app or service in the best way possible to use these parameters.
Additionally, is it running on the appropriate generation of hardware? Should it run on newer or older gen hardware? Do you need EBS Optimized, is it EBS optimized? Which is more cost-efficient, are there any traits we can take advantage of?
T class instances generate and consume CPU credits during their usage, this is great for workloads that are used only occasionally but can become problematic or expensive when utilization is steady or high.
Each instance type generally comes with a range of “sizes”. These allocate more CPU, Memory, Network, and sometimes impact storage options. Are we over/under-provisioned? Most instances are vastly underutilized and lots of savings can be accomplished by reducing the instance sizes or changing the strategy (covered in next section).
Right count, scaling appropriately
What is your strategy for managing load scaling for this workload?
Vertically scaling (big iron)
Leads to fewer large instance classes. Traits of vertical scaling:
- Prevalent with a legacy application (monolith), databases, and operations with large datasets
- Employed to reduce licensing costs for outdated licensing models (host count)
- More often have static configuration values (ip addresses, names, etc)
- More often Microsoft Windows-based
- Often treated as Pets
- The high impact of an instance failure
- Many services/apps on a single host (sometimes unrelated)
Capacity changes happen in large increments (hard to match to utilization)
An approach that will use many smaller servers to satisfy SLA and load requirements. Traits of horizontal scaling:
- Prevalent with tiered, service-based, and microservice architecture
- More typically Linux based
- More often leverage some degree of automation
- Often treated as Cattle
- Low impact of instance failure
- Oriented at a single service or function
Better leverages the benefits of the cloud, when combined with automation can allow higher degree of cost control.
Our Recommendations on Right Sizing
There are generally few scenarios where larger instances are the more efficient option, though they do exist. We recommend trying to match your capacity to your utilization as closely as possible to avoid over-spending. The smaller scaling steps you can take
Right usage (pets vs cattle)
How do you map your needs to your hosts? Do you have 1 function and assign a fleet of hosts, or do you fit and route services into whatever hosts are available with extra capacity? We’re going to divide this into 2 camps called Pets and Cattle.
Containers and Functions
The guidance so far has been focused EC2 instances but there are other ways to satisfy compute requirements in AWS.
Containers and container tooling is very popular right now as a method to increase utilization on resources by bounding the work vessel more tightly to the workload with less overhead. In most formats you must pay for the host the container is running on but scheduling services help you fill those hosts more densely.
Lambda Functions on the other hand askew the concept of paying for something you aren’t using and let you scale to 0 (reserved concurrency breaks this model). They do this by allowing you to ship your code and choose your runtime, you then invoke the function with an event and a context (your input) and the lambda will do what it’s been programmed to do.
Leverage both of these patterns where possible to increase utilization. For glue tasks like cron, event-driven, or administrative tooling Lambda functions are a natural fit. If you have more control over your workload architecture and code, identify tasks like batch processing, report generation, etc. that you can shift into Lambda or Fargate.
Focusing the types of work your EC2 instances do creates more opportunities for optimization.