Industry research and customer analysis shows that on average, 20% of files are actively used and 80% are infrequently accessed. AWS EFS Infrequent Access (IA) storage costs only 8.3% ($0.025/Gb-Month) of the cost of AWS EFS Standard storage ($0.3/Gb-Month). These numbers are even lower for One Zone storage. By migrating infrequently used files to Infrequent Access storage, you can save almost 92% in storage costs.
Enabling AWS EFS Intelligent-Tiering will use lifecycle management to monitor your workload access patterns and automatically transition files that are not accessed for a specified duration (e.g. 30 days) to Infrequent Access storage.
Background: AWS storage types and pricing
AWS offers multiple storage options:
- Elastic File System (EFS)
- Elastic Block Storage (EBS)
- Simple Storage Service (S3)
Both EFS and EBS offer file system access semantics (such as strong data consistency and file locking), and can, depending on the operating system, be managed using both POSIX and AWS IAM permissions. AWS S3, on the other hand, does not directly offer filesystem access semantics; if this access is necessary for your application, you can only choose between EFS and EBS.
- EFS cannot be used for OS boot partitions, so if you need to boot an EC2 instance you must use EBS.
- EFS is slower (up to 2x slower to first byte) than EBS and can be affected by network latency. It is not a good choice for mounting app code (e.g. PHP applications etc.)
- However, only EFS can be accessed by 1000s of instances/services simultaneously using file system semantics. Multi-attach enabled EBS volumes can only be attached to a maximum of 16 EC2 instances at a time.
Typical pricing
| Storage | Price/Gb-Month | Notes |
|---|---|---|
| S3 | $0.023 (S3 IA: $0.0125/Gb) | Least expensive, can be even lower if using Infrequent Access. |
| EFS (One Zone) | One Zone: $0.160; $0.0133: IA. Effectively | Standard (multi-AZ) storage is approx 2x the price of One Zone storage. (EFS IA Storage is 16% of equivalent EBS Storage) |
| EBS | $0.080 (SSD gp3) | A multi-attach enabled volume only can be attached to a max of only 16 EC2 Instances |
You can configure your AWS EFS volumes to use either Standard and One Zone storage classes. AWS EFS offers you two performance modes and two throughput modes. Your selection of these two parameters will drive the performance and pricing of your AWS EFS file system.
Should you use Standard or One Zone Storage?
Standard Storage:
- This is the default option. Data is redundantly stored across multiple availability zones to provide the greatest availability.
One Zone Storage:
- One Zone storage redundantly stores your data in only one AZ. It has 11 9s durability equivalent to Standard storage.
- The typical recommendation is to use One Zone storage for data that can be regenerated (e.g. development, build, and staging environments, or analytics, simulation and media transcoding), and for backups or replicas of on-premises data that does not require Multi-AZ resilience.
- However, in the entire outage history of AWS there has never been an EFS specific outage. We use One Zone storage for all our EFS needs, and you should make a considered evaluation of whether this is an appropriate choice for your needs. Further, when creating a One Zone file system, automatic backups are enabled by default, providing daily backups with a 35-day retention period.
- One Zone storage is 47% less costly than Standard storage.
- You can’t migrate an existing file system between One Zone and Standard storage.
What type of AWS EFS Storage should you select?
AWS offers two performance modes for EFS:
- General Purpose performance mode: General Purpose is the default mode, and is ideal for latency-sensitive use cases, like web serving environments, content management systems, home directories, and general file serving.
- Max I/O performance mode: Max I/O mode can scale to higher levels of aggregate throughput and operations per second. Highly parallelized applications and workloads, such as big data analysis, media processing, and genomic analysis will benefit from this mode.
How to select a throughput mode
AWS offers two throughout modes:
- Bursting Throughput mode: This is the default, and allows a maximum throughput based on the size of your EFS file system, typically a minimum of approx 100Mbps for 72 minutes. This burst rate almost always allows transfer of all data in the file system per day. This mode, with a few exceptions, works well with the General Purpose performance mode use cases.
- Provisioned Throughput mode: This throughput mode works best for applications with high throughput-to-storage ratios such as media processing (for which you should use Max I/O performance mode).
How to reduce costs by enabling lifecycle management
Infrequent Access (IA) costs only 8% of the cost of Standard storage. IA has a slightly higher latency to first byte compared to Standard storage. Lifecycle management automates the process of moving your infrequently accessed files to IA storage, and bringing them back when they are accessed. Here is how to enable lifecycle management:
- Open the AWS EFS Management Console and select the file systems that you want to manage.
- On the file system details page choose Edit >> General settings page, which will show you:
- For lifecycle management, you can change the following lifecycle policies:
- Set Transition into IA to one of the available settings.
- Set Transition out of IA to On first access to move files that are in IA storage to standard storage when they’re accessed for non-metadata operations.
- You have now enabled EFS lifecycle management.
If you don’t want to manually apply this fix across all your file systems, try CloudFix, a simple automated solution for finding and implementing AWS best practices.