Skip to main content

AWS Storage Services

Overview​

AWS provides multiple storage services for different use cases. This guide focuses on EBS (Elastic Block Store) for block-level storage and EFS (Elastic File System) for shared file storage. For object storage, see the dedicated S3 guide.

EBS provides persistent block storage volumes that attach to EC2 instances, functioning as virtual hard drives for databases and boot volumes. EFS provides managed NFS file systems that multiple instances can mount simultaneously, enabling shared access across applications and containers.

Core Principles​

  1. EBS for single-instance workloads - Databases, boot volumes, applications requiring low-latency block access
  2. EFS for shared access - Container storage, shared file systems, multi-instance applications
  3. Design for failure - Use Multi-AZ, snapshots, and automated backups
  4. Encrypt everything - Enable encryption at rest for all volumes and file systems
  5. Monitor performance - Track IOPS, throughput, and capacity to identify bottlenecks

Storage Service Selection​

When to use each service:

EBS - Block storage for single EC2 instances:

  • Database storage (PostgreSQL, MySQL, Oracle)
  • Boot volumes for EC2 instances
  • Applications requiring consistent low-latency I/O
  • High IOPS workloads (up to 256,000 IOPS)

EFS - Shared NFS file system for multiple instances:

  • Container storage (EKS/ECS shared volumes)
  • Content management systems with multiple servers
  • Shared development environments
  • Machine learning training data

S3 - Object storage for web-scale applications (see AWS S3):

  • User-generated content (photos, videos, documents)
  • Application backups and archives
  • Static website hosting
  • Data lakes

EBS (Elastic Block Store)​

EBS provides block-level storage volumes that attach to EC2 instances. Unlike instance store (ephemeral storage), EBS volumes persist independently and can be detached/reattached to different instances.

EBS volumes exist within a single Availability Zone. While this provides low-latency access, volumes cannot be directly attached to instances in other AZs. For Multi-AZ architectures, use snapshots to copy volumes across AZs.

Volume Types and Performance​

EBS offers multiple volume types optimized for different workloads. Selecting the appropriate type balances performance and cost.

gp3 (General Purpose SSD) - Default choice for most workloads:

  • Base performance: 3,000 IOPS and 125 MB/s regardless of size
  • Scalability: Scale IOPS (up to 16,000) and throughput (up to 1,000 MB/s) independently
  • Cost: $0.08/GB-month + additional for IOPS/throughput above baseline
  • Use cases: Boot volumes, development environments, moderate-performance databases
  • Why better than gp2: gp2 performance scales with size (3 IOPS per GB), forcing you to over-provision capacity for IOPS. gp3 decouples capacity from performance.

io2 (Provisioned IOPS SSD) - Latency-sensitive production workloads:

  • Performance: Up to 64,000 IOPS per volume (256,000 with io2 Block Express)
  • Durability: 99.999% (5 9's) vs 99.8-99.9% for gp3
  • Cost: $0.125/GB-month + $0.065 per provisioned IOPS-month
  • Use cases: Mission-critical databases (Oracle, SQL Server, PostgreSQL), NoSQL requiring consistent latency
  • Multi-Attach: io2 supports attaching to up to 16 Nitro instances simultaneously

st1 (Throughput Optimized HDD) - Sequential, throughput-intensive workloads:

  • Performance: Baseline 40 MB/s per TB, burst to 250 MB/s per TB, max 500 MB/s
  • Cost: $0.045/GB-month (half the cost of gp3)
  • Use cases: Big data, log aggregation, data warehouses
  • Not suitable for: Random I/O, databases, boot volumes (cannot boot from HDD)

sc1 (Cold HDD) - Lowest cost for infrequent access:

  • Performance: Baseline 12 MB/s per TB, burst to 80 MB/s per TB, max 250 MB/s
  • Cost: $0.015/GB-month (5x cheaper than gp3)
  • Use cases: Archival storage, cold data
  • Not suitable for: Active workloads, databases, boot volumes
// Spring Boot configuration for EBS volume verification
@Configuration
public class EBSVolumeConfig {

    private static final Logger log = LoggerFactory.getLogger(EBSVolumeConfig.class);

    @Value("${ebs.mount.path:/mnt/data}")
    private String mountPath;

    @PostConstruct
    public void verifyEBSMount() {
        File dataDir = new File(mountPath);

        if (!dataDir.exists()) {
            throw new IllegalStateException(
                "EBS volume not mounted at: " + mountPath);
        }

        if (!dataDir.canWrite()) {
            throw new IllegalStateException(
                "No write permission for EBS mount: " + mountPath);
        }

        long usableSpace = dataDir.getUsableSpace();
        long totalSpace = dataDir.getTotalSpace();
        double usagePercent = ((totalSpace - usableSpace) * 100.0) / totalSpace;

        log.info("EBS volume mounted at {} with {:.1f}% used ({} GB available)",
            mountPath, usagePercent, usableSpace / (1024 * 1024 * 1024));

        if (usagePercent > 85) {
            log.warn("EBS volume usage exceeds 85%, consider expanding");
        }
    }
}

Choosing between gp3 and io2: Use gp3 for most workloads3,000 IOPS baseline satisfies 90% of applications. Choose io2 when you need:

  1. More than 16,000 IOPS per volume
  2. Higher durability (5 9's vs 99.8%)
  3. Multi-Attach for clustered applications
  4. Sub-millisecond latency consistency under load

EBS Snapshots​

Snapshots create point-in-time backups stored in S3. Snapshots are incrementalonly changed blocks are saved after the first snapshot.

Snapshot workflow:

  1. First snapshot copies all data to S3
  2. Subsequent snapshots copy only changed blocks
  3. Deleting intermediate snapshots doesn't affect restore capability (AWS maintains block references)
// Automated snapshot creation with retention
@Service
public class EBSSnapshotService {

    private final Ec2Client ec2Client;

    @Scheduled(cron = "0 0 2 * * *")  // Daily at 2 AM
    public void createDailySnapshots() {
        DescribeVolumesResponse volumes = ec2Client.describeVolumes(
            DescribeVolumesRequest.builder()
                .filters(Filter.builder()
                    .name("tag:Backup")
                    .values("daily")
                    .build())
                .build()
        );

        for (Volume volume : volumes.volumes()) {
            String description = String.format(
                "Daily backup of %s on %s",
                volume.volumeId(),
                LocalDate.now()
            );

            CreateSnapshotResponse snapshot = ec2Client.createSnapshot(
                CreateSnapshotRequest.builder()
                    .volumeId(volume.volumeId())
                    .description(description)
                    .tagSpecifications(TagSpecification.builder()
                        .resourceType(ResourceType.SNAPSHOT)
                        .tags(
                            Tag.builder().key("Name").value("daily-backup").build(),
                            Tag.builder().key("VolumeId").value(volume.volumeId()).build(),
                            Tag.builder().key("CreatedDate").value(LocalDate.now().toString()).build()
                        )
                        .build())
                    .build()
            );

            log.info("Created snapshot {} for volume {}",
                snapshot.snapshotId(), volume.volumeId());
        }

        deleteExpiredSnapshots(30);  // 30-day retention
    }

    private void deleteExpiredSnapshots(int retentionDays) {
        LocalDate cutoffDate = LocalDate.now().minusDays(retentionDays);

        DescribeSnapshotsResponse snapshots = ec2Client.describeSnapshots(
            DescribeSnapshotsRequest.builder()
                .ownerIds("self")
                .filters(Filter.builder()
                    .name("tag:Name")
                    .values("daily-backup")
                    .build())
                .build()
        );

        for (Snapshot snapshot : snapshots.snapshots()) {
            Instant snapshotTime = snapshot.startTime();
            LocalDate snapshotDate = LocalDate.ofInstant(snapshotTime, ZoneId.systemDefault());

            if (snapshotDate.isBefore(cutoffDate)) {
                ec2Client.deleteSnapshot(DeleteSnapshotRequest.builder()
                    .snapshotId(snapshot.snapshotId())
                    .build());

                log.info("Deleted expired snapshot {} from {}",
                    snapshot.snapshotId(), snapshotDate);
            }
        }
    }
}

Snapshot strategies:

  • Daily snapshots with 7-30 day retention for active production volumes
  • Weekly snapshots with 90-day retention for less critical data
  • Cross-region snapshot copies for disaster recovery
  • AWS Data Lifecycle Manager (DLM) to automate creation, retention, and deletion

Snapshot costs: You pay for total storage used, not per snapshot. If you have a 100 GB volume with 5 snapshots and only 20 GB changed, you pay for 120 GB, not 500 GB. Snapshots are compressed and deduplicated automatically.

For backup strategies, see Disaster Recovery.

EBS Encryption​

All EBS volumes should be encrypted. Encryption protects data at rest, in transit between EC2 and EBS, and all snapshots.

Enable default encryption for all new volumes:

# Enable default encryption in a region
aws ec2 enable-ebs-encryption-by-default --region us-east-1

Application-level validation:

// Verify EBS volume encryption
@Configuration
public class EBSEncryptionValidator {

    private final Ec2Client ec2Client;

    @Value("${ebs.volume.id}")
    private String volumeId;

    @PostConstruct
    public void validateEncryption() {
        DescribeVolumesResponse response = ec2Client.describeVolumes(
            DescribeVolumesRequest.builder()
                .volumeIds(volumeId)
                .build()
        );

        Volume volume = response.volumes().get(0);

        if (!volume.encrypted()) {
            throw new IllegalStateException(
                "EBS volume " + volumeId + " is not encrypted. " +
                "All production volumes must be encrypted."
            );
        }

        log.info("EBS volume {} encrypted with KMS key {}",
            volumeId, volume.kmsKeyId());
    }
}

Encryption performance: Nitro-based EC2 instances have dedicated hardware for encryption with negligible overhead. Never disable encryption for performance on Nitro instances.

Migrating unencrypted to encrypted volumes:

  1. Create snapshot of unencrypted volume
  2. Copy snapshot with encryption enabled
  3. Create encrypted volume from encrypted snapshot
  4. Attach encrypted volume to instance
  5. Delete old unencrypted volume

For KMS key management, see Secrets Management.

EBS Performance Optimization​

EBS-optimized instances provide dedicated bandwidth for EBS I/O separate from network traffic. All current-generation instances are EBS-optimized by default.

Volume initialization: Volumes created from snapshots load blocks lazily from S3 on first access, causing high latency. Pre-warm volumes for production databases:

# Pre-warm EBS volume from snapshot (Linux)
sudo fio --filename=/dev/nvme1n1 --rw=read --bs=128k --iodepth=32 --ioengine=libaio --direct=1 --name=volume-init

This forces all blocks to load from S3, eliminating lazy-load latency.

Performance monitoring:

// Monitor EBS performance metrics
@Service
public class EBSMetricsService {

    private final CloudWatchClient cloudWatchClient;
    private final MeterRegistry meterRegistry;

    @Scheduled(fixedRate = 60000)  // Every minute
    public void collectEBSMetrics() {
        String volumeId = System.getenv("EBS_VOLUME_ID");
        Instant endTime = Instant.now();
        Instant startTime = endTime.minus(5, ChronoUnit.MINUTES);

        // Get VolumeReadOps
        GetMetricStatisticsResponse readOps = cloudWatchClient.getMetricStatistics(
            GetMetricStatisticsRequest.builder()
                .namespace("AWS/EBS")
                .metricName("VolumeReadOps")
                .dimensions(Dimension.builder()
                    .name("VolumeId")
                    .value(volumeId)
                    .build())
                .startTime(startTime)
                .endTime(endTime)
                .period(300)
                .statistics(Statistic.SUM)
                .build()
        );

        double iops = readOps.datapoints().stream()
            .mapToDouble(Datapoint::sum)
            .average()
            .orElse(0) / 300.0;

        meterRegistry.gauge("ebs.volume.read.iops", iops);

        double provisionedIOPS = getProvisionedIOPS(volumeId);
        if (iops > provisionedIOPS * 0.8) {
            log.warn("EBS volume {} at {:.1f}% of provisioned IOPS ({:.0f}/{:.0f})",
                volumeId, (iops / provisionedIOPS) * 100, iops, provisionedIOPS);
        }
    }

    private double getProvisionedIOPS(String volumeId) {
        DescribeVolumesResponse response = ec2Client.describeVolumes(
            DescribeVolumesRequest.builder()
                .volumeIds(volumeId)
                .build()
        );

        Volume volume = response.volumes().get(0);
        return volume.iops() != null ? volume.iops() : 3000;  // gp3 baseline
    }
}

For observability patterns, see Observability and AWS Observability.

EBS Multi-Attach​

io2 volumes support Multi-Attach, allowing up to 16 Nitro instances in the same AZ to attach simultaneously. This enables clustered applications without application-level replication.

Use cases:

  • Clustered databases (Oracle RAC, SQL Server failover clusters)
  • Shared storage for high-availability applications
  • Concurrent data access by multiple nodes

Limitations:

  • Only io2 volumes with Nitro instances
  • All instances must be in same AZ
  • Application must implement file locking (EBS doesn't provide it)
  • Requires cluster-aware file system (GFS2, OCFS2)

Multi-Attach is complex and requires application coordination. For most shared storage needs, EFS provides simpler multi-instance access.

EFS (Elastic File System)​

EFS provides fully managed NFS file systems that multiple EC2 instances and containers can mount simultaneously. EFS automatically scales capacity and you pay only for storage used.

EFS is built for high availability by storing data redundantly across multiple AZs. This Multi-AZ architecture ensures data remains available even if an entire AZ fails.

Performance Modes and Throughput​

EFS offers two performance modes (selected at creation, cannot be changed):

General Purpose - Default for most workloads:

  • Lower latency (sub-millisecond for file operations)
  • Up to 35,000 operations per second
  • Suitable for web serving, CMS, home directories, development

Max I/O - Massive parallelism:

  • Higher aggregate throughput (500,000+ ops/sec)
  • Slightly higher latency (low single-digit milliseconds)
  • Suitable for big data analytics, media processing, genomics with thousands of concurrent clients

Throughput modes:

Bursting - Throughput scales with file system size:

  • 50 MB/s per TB baseline
  • Burst to 100 MB/s for file systems > 1 TB
  • Burst credits accumulate when below baseline
  • Suitable for intermittent access patterns

Provisioned - Fixed throughput independent of size:

  • Specify desired throughput (1-1,024 MB/s)
  • Pay additional per MB/s-month
  • Suitable for consistent high-throughput with small storage

Elastic - Recommended for unpredictable workloads:

  • Auto-scales up to 3 GB/s reads, 1 GB/s writes
  • Pay per GB transferred
  • Ideal for spiky workloads or difficult capacity planning

Choosing modes: Start with General Purpose + Bursting. Monitor PercentIOLimit CloudWatch metricif consistently >80%, switch to Provisioned or Elastic. For big data with thousands of workers, use Max I/O.

EFS with EKS/ECS​

EFS is the recommended shared storage for containerized applications. Multiple pods/tasks can mount the same file system.

EFS CSI Driver for Kubernetes:

# StorageClass for EFS
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: efs-sc
provisioner: efs.csi.aws.com
parameters:
  provisioningMode: efs-ap
  fileSystemId: fs-0123456789abcdef0
  directoryPerms: "700"
  gidRangeStart: "1000"
  gidRangeEnd: "2000"
  basePath: "/dynamic_provisioning"

---
# PersistentVolumeClaim
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: efs-claim
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: efs-sc
  resources:
    requests:
      storage: 5Gi  # Size is symbolic; EFS auto-scales

---
# Pod using EFS
apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  containers:
    - name: app
      image: my-app:latest
      volumeMounts:
        - name: persistent-storage
          mountPath: /data
  volumes:
    - name: persistent-storage
      persistentVolumeClaim:
        claimName: efs-claim

EFS benefits for Kubernetes:

  • ReadWriteMany (RWX) - Multiple pods across nodes can mount simultaneously
  • Automatic scaling - No capacity planning needed
  • Multi-AZ durability - Data survives AZ failures
  • Access Points - Isolated namespaces for multi-tenancy

For Kubernetes storage patterns, see Kubernetes and AWS EKS.

EFS with ECS:

{
  "family": "my-task",
  "networkMode": "awsvpc",
  "containerDefinitions": [
    {
      "name": "app",
      "image": "my-app:latest",
      "mountPoints": [
        {
          "sourceVolume": "efs-storage",
          "containerPath": "/mnt/efs",
          "readOnly": false
        }
      ]
    }
  ],
  "volumes": [
    {
      "name": "efs-storage",
      "efsVolumeConfiguration": {
        "fileSystemId": "fs-0123456789abcdef0",
        "transitEncryption": "ENABLED",
        "authorizationConfig": {
          "accessPointId": "fsap-0123456789abcdef0",
          "iam": "ENABLED"
        }
      }
    }
  ]
}

Mount targets: Network endpoints in each AZ's subnet allowing instances to mount the file system. Create one mount target per AZ for high availability. See AWS Networking for VPC configuration.

EFS Lifecycle Management​

Lifecycle Management automatically transitions infrequently accessed files to lower-cost storage classes.

EFS storage classes:

Standard - Frequently accessed:

  • $0.30/GB-month
  • Sub-millisecond latency
  • Default storage class

Infrequent Access (IA) - Files not accessed for 7/14/30/60/90 days:

  • $0.025/GB-month (1/12 the cost of Standard)
  • $0.01/GB retrieval fee
  • Slightly higher latency on first access
  • Files automatically move back to Standard when accessed

Archive - Long-term storage:

  • $0.016/GB-month (1/19 the cost of Standard)
  • $0.03/GB retrieval fee
  • Use for compliance archives, old backups

Enable lifecycle policies:

@Configuration
public class EFSLifecycleConfig {

    private final EfsClient efsClient;

    @Value("${efs.filesystem.id}")
    private String fileSystemId;

    @PostConstruct
    public void configureLifecycle() {
        efsClient.putLifecycleConfiguration(
            PutLifecycleConfigurationRequest.builder()
                .fileSystemId(fileSystemId)
                .lifecyclePolicies(
                    // Transition to IA after 30 days
                    LifecyclePolicy.builder()
                        .transitionToIA(TransitionToIARules.AFTER_30_DAYS)
                        .build(),
                    // Transition to Archive after 90 days in IA
                    LifecyclePolicy.builder()
                        .transitionToArchive(TransitionToArchiveRules.AFTER_90_DAYS)
                        .build()
                )
                .build()
        );

        log.info("Configured lifecycle policies for EFS {}", fileSystemId);
    }
}

Cost optimization: For workloads with predictable access (e.g., logs stored for compliance), lifecycle policies can reduce costs by 80-90%. A 1 TB file system where 80% hasn't been accessed in 30 days: 200 GB Standard ($60) + 800 GB IA ($20) = $80/month vs $300/month without lifecycle management.

EFS Encryption and Access Control​

EFS supports encryption at rest and in transit. Always enable both for production.

Encryption at rest (configured at creation):

# Create encrypted EFS
aws efs create-file-system \
  --encrypted \
  --kms-key-id arn:aws:kms:us-east-1:123456789012:key/abcd1234 \
  --performance-mode generalPurpose \
  --throughput-mode elastic \
  --tags Key=Name,Value=my-efs

Encryption in transit (enabled when mounting):

# Mount with TLS encryption
sudo mount -t efs -o tls fs-0123456789abcdef0:/ /mnt/efs

For containers, enable transitEncryption: ENABLED in ECS task definitions or EFS CSI driver.

EFS Access Points provide application-specific entry points with enforced user identity and root directory:

@Service
public class EFSAccessPointService {

    private final EfsClient efsClient;

    public String createAccessPoint(String fileSystemId, String appName, int uid, int gid) {
        CreateAccessPointResponse response = efsClient.createAccessPoint(
            CreateAccessPointRequest.builder()
                .fileSystemId(fileSystemId)
                .posixUser(PosixUser.builder()
                    .uid(uid)
                    .gid(gid)
                    .build())
                .rootDirectory(RootDirectory.builder()
                    .path("/" + appName)
                    .creationInfo(CreationInfo.builder()
                        .ownerUid(uid)
                        .ownerGid(gid)
                        .permissions("755")
                        .build())
                    .build())
                .tags(Tag.builder().key("Application").value(appName).build())
                .build()
        );

        log.info("Created EFS access point {} for {} with UID/GID {}/{}",
            response.accessPointId(), appName, uid, gid);

        return response.accessPointId();
    }
}

Access Points ensure containers/instances can only access their designated directory with enforced permissions.

For IAM-based access control, see AWS IAM and Authorization.

Backup and Disaster Recovery​

Comprehensive backup strategy is critical for business continuity.

EBS Snapshots:

  • Incremental backups to S3
  • Cross-region copying for geographic redundancy
  • AWS Data Lifecycle Manager for automation
  • Point-in-time recovery

EFS Backups:

  • Incremental backups to S3 with AWS Backup
  • Automatic scheduling (daily, weekly, monthly)
  • Cross-region backup for DR
  • Restore to same or different region

AWS Backup - Centralized backup management:

@Configuration
public class AWSBackupConfig {

    private final BackupClient backupClient;

    public void createBackupPlan() {
        BackupPlanInput planInput = BackupPlanInput.builder()
            .backupPlanName("production-daily-backup")
            .rules(BackupRuleInput.builder()
                .ruleName("daily-backup")
                .targetBackupVaultName("production-vault")
                .scheduleExpression("cron(0 2 * * ? *)")  // 2 AM daily
                .startWindowMinutes(60L)
                .completionWindowMinutes(120L)
                .lifecycle(Lifecycle.builder()
                    .deleteAfterDays(30L)
                    .moveToColdStorageAfterDays(7L)
                    .build())
                .build())
            .build();

        CreateBackupPlanResponse response = backupClient.createBackupPlan(
            CreateBackupPlanRequest.builder()
                .backupPlan(planInput)
                .build()
        );

        // Assign resources via tags
        backupClient.createBackupSelection(
            CreateBackupSelectionRequest.builder()
                .backupPlanId(response.backupPlanId())
                .backupSelection(BackupSelection.builder()
                    .selectionName("production-resources")
                    .iamRoleArn("arn:aws:iam::123456789012:role/AWSBackupRole")
                    .listOfTags(Condition.builder()
                        .conditionType(ConditionType.STRINGEQUALS)
                        .conditionKey("Backup")
                        .conditionValue("daily")
                        .build())
                    .build())
                .build()
        );
    }
}

RTO/RPO:

  • EBS snapshots: RPO = snapshot frequency, RTO = 5-15 minutes
  • EFS backups: RPO = backup frequency, RTO = 10-30 minutes

For disaster recovery strategies, see Disaster Recovery.

Storage Cost Optimization​

EBS optimization:

  1. Right-size volumes (provision only needed capacity)
  2. Delete unused snapshots
  3. Use gp3 instead of gp2 (20% cheaper)
  4. Delete detached volumes
  5. Use snapshot lifecycle policies

EFS optimization:

  1. Enable lifecycle management (transition to IA/Archive)
  2. Use Elastic throughput mode
  3. Monitor access patterns
  4. Consolidate file systems
  5. Implement retention policies

For comprehensive cost optimization, see AWS Cost Optimization.

Anti-Patterns​

Using EBS for shared storage: EBS attaches to single instances. For shared access, use EFS or S3.

Not enabling encryption: All storage should be encrypted. Compliance violations cost more than negligible encryption overhead.

Over-provisioning IOPS: Provisioning 10,000 IOPS when needing 2,000 wastes $260/month. Monitor and adjust.

Using EFS for databases: EFS has higher latency than EBS. Use EBS for databases or managed services (RDS, Aurora).

Storing application state on instance store: Instance store is ephemeral. Use EBS, EFS, or S3 for persistent data.

Not implementing lifecycle policies: Old data in expensive tiers wastes money. Automate transitions.

Ignoring snapshot costs: Snapshots accumulate. Delete obsolete backups automatically.

Not testing restores: Backups are useless if you can't restore. Test restore procedures regularly.

Further Reading​