Kubernetes EFS Storage (IAM Role)
Luke Hebert
- 5 minutes read - 945 wordsChoosing between EFS and S3 for Kubernetes workloads comes down to your application’s storage needs. EFS provides a POSIX-compliant filesystem with ReadWriteMany access, making it ideal when multiple pods need concurrent read/write access, file locking, or traditional file operations. S3, on the other hand, suits object storage patterns, archival data, and cost-sensitive workloads with infrequent access. The setup below walks through provisioning an EFS filesystem, configuring IAM roles and security groups, and deploying Kubernetes manifests (StorageClass, PVCs, Pods, Deployments, and StatefulSets) that leverage the AWS EFS CSI driver for shared, elastic storage.
Key Differences: EFS vs S3 Storage
EFS Advantages
- True Filesystem: POSIX-compliant, supports file locking, permissions
- ReadWriteMany: Multiple pods can read/write simultaneously
- Performance: Better for frequent small file operations
- Caching: Built-in caching for better performance
- Elastic: Automatically scales storage capacity
Key Components
- ServiceAccount: Binds an IAM role to the pod for secure EFS access
- StorageClass: Defines how EFS volumes should be provisioned using the EFS CSI driver
- PersistentVolumeClaim: Requests storage from the EFS StorageClass
- Pod/Deployment: Shows how to mount and use the EFS storage
- CSI Driver Required: You’ll need to install the AWS EFS CSI driver in your cluster first
- Credentials: Use an IAM role attached to the pod’s service account rather than hardcoded credentials
- Filesystem Configuration: Update the filesystem ID and region to match your EFS setup
- Region: Use the appropriate AWS region for your EFS filesystem
Installation of the EFS CSI Driver
kubectl apply -k "github.com/kubernetes-sigs/aws-efs-csi-driver/deploy/kubernetes/overlays/stable/?ref=release-1.7"Or via Helm:
helm repo add aws-efs-csi-driver https://kubernetes-sigs.github.io/aws-efs-csi-driver/
helm install aws-efs-csi-driver aws-efs-csi-driver/aws-efs-csi-driver1. Create EFS Filesystem
# Create EFS filesystem
aws efs create-file-system \
--region us-west-2 \
--performance-mode generalPurpose \
--throughput-mode provisioned \
--provisioned-throughput-in-mibps 500 \
--tags Key=Name,Value=k8s-efs
# Create mount targets in your VPC subnets
aws efs create-mount-target \
--file-system-id fs-0123456789abcdef0 \
--subnet-id subnet-12345 \
--security-groups sg-abcdef1232. Create IAM Role for EFS
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"elasticfilesystem:CreateAccessPoint",
"elasticfilesystem:DeleteAccessPoint",
"elasticfilesystem:DescribeAccessPoints",
"elasticfilesystem:DescribeFileSystems"
],
"Resource": "*"
}
]
}3. Security Group Configuration
Your EFS mount targets need a security group that allows NFS traffic:
# Allow NFS traffic (port 2049) from your EKS worker nodes
aws ec2 authorize-security-group-ingress \
--group-id sg-efs123 \
--protocol tcp \
--port 2049 \
--source-group sg-eks-nodes123Key Manifest Features
- Dynamic Provisioning: StorageClass creates EFS access points automatically
- Static Volumes: Direct reference to existing EFS filesystem
- Multiple Access Modes: ReadWriteMany for shared storage
- StatefulSet Example: Shows both shared (EFS) and per-instance (EBS) storage
- IAM Role Integration: Secure access without hardcoded credentials
When to Use EFS vs S3
Use EFS when:
- You need POSIX filesystem semantics
- Multiple pods need concurrent read/write access
- Your application expects traditional file operations
- You need file locking or permissions
Use S3 when:
- You have object storage patterns
- You’re storing archival or backup data
- You’re handling large files with infrequent access
- Cost is the primary concern
Complete Manifest Example
# Service Account with IAM role annotation for EFS
apiVersion: v1
kind: ServiceAccount
metadata:
name: efs-service-account
namespace: default
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/EFSAccessRole
---
# Storage Class for EFS
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: efs-storage-class
provisioner: efs.csi.aws.com
parameters:
provisioningMode: efs-ap # Creates access points automatically
fileSystemId: fs-0123456789abcdef0 # Your EFS filesystem ID
directoryPerms: "0755"
gidRangeStart: "1000"
gidRangeEnd: "2000"
basePath: "/dynamic_provisioning" # Base path for access points
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: Immediate
---
# Persistent Volume Claim for EFS
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: efs-pvc
namespace: default
spec:
accessModes:
- ReadWriteMany # EFS supports multiple readers/writers
storageClassName: efs-storage-class
resources:
requests:
storage: 5Gi # Note: EFS doesn't enforce size limits, this is just metadata
---
# Static Persistent Volume for existing EFS (alternative approach)
apiVersion: v1
kind: PersistentVolume
metadata:
name: efs-static-pv
spec:
capacity:
storage: 5Gi
volumeMode: Filesystem
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Retain
storageClassName: efs-static
csi:
driver: efs.csi.aws.com
volumeHandle: fs-0123456789abcdef0 # Your EFS filesystem ID
# Optional: Use access point
# volumeHandle: fs-0123456789abcdef0::fsap-01234567890abcdef
---
# PVC for static PV
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: efs-static-pvc
namespace: default
spec:
accessModes:
- ReadWriteMany
storageClassName: efs-static
resources:
requests:
storage: 5Gi
---
# Example Pod using EFS storage
apiVersion: v1
kind: Pod
metadata:
name: efs-pod
namespace: default
spec:
serviceAccountName: efs-service-account
containers:
- name: app-container
image: nginx:latest
volumeMounts:
- name: efs-volume
mountPath: /data
# No need for explicit AWS credentials - using IAM role
volumes:
- name: efs-volume
persistentVolumeClaim:
claimName: efs-pvc
---
# Direct EFS mount (without PVC)
apiVersion: v1
kind: Pod
metadata:
name: efs-direct-pod
namespace: default
spec:
serviceAccountName: efs-service-account
containers:
- name: app-container
image: nginx:latest
volumeMounts:
- name: efs-direct-volume
mountPath: /efs-data
volumes:
- name: efs-direct-volume
csi:
driver: efs.csi.aws.com
volumeHandle: fs-0123456789abcdef0 # Your EFS filesystem ID
---
# Deployment with EFS storage
apiVersion: apps/v1
kind: Deployment
metadata:
name: efs-app-deployment
namespace: default
spec:
replicas: 3
selector:
matchLabels:
app: efs-app
template:
metadata:
labels:
app: efs-app
spec:
serviceAccountName: efs-service-account
containers:
- name: app
image: nginx:latest
ports:
- containerPort: 80
volumeMounts:
- name: efs-storage
mountPath: /var/www/html
- name: efs-logs
mountPath: /var/log/nginx
# No need for explicit AWS credentials - using IAM role
readinessProbe:
httpGet:
path: /
port: 80
initialDelaySeconds: 5
periodSeconds: 10
livenessProbe:
httpGet:
path: /
port: 80
initialDelaySeconds: 15
periodSeconds: 20
volumes:
- name: efs-storage
persistentVolumeClaim:
claimName: efs-pvc
- name: efs-logs
persistentVolumeClaim:
claimName: efs-static-pvc
---
# StatefulSet example with EFS (shared) and EBS (per-instance)
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: efs-statefulset
namespace: default
spec:
serviceName: efs-service
replicas: 3
selector:
matchLabels:
app: efs-stateful
template:
metadata:
labels:
app: efs-stateful
spec:
serviceAccountName: efs-service-account
containers:
- name: app
image: nginx:latest
ports:
- containerPort: 80
volumeMounts:
- name: shared-efs
mountPath: /shared # Shared across all replicas
- name: instance-data
mountPath: /data # Per-instance storage
volumes:
- name: shared-efs
persistentVolumeClaim:
claimName: efs-pvc
volumeClaimTemplates:
- metadata:
name: instance-data
spec:
accessModes:
- ReadWriteOnce
storageClassName: gp2 # Regular EBS for per-instance data
resources:
requests:
storage: 1Gi