Monday, 12 June 2017

Creating an Elastic File System (EFS) within AWS

The EFS is the AWS equivalent of network attached storage. It works on NFSv4 protocol. Under the free tier we can use up to 5GB of EFS storage per month. As of this writing EFS is available for Linux only. The initial size of an EFS is quite large and can grow or shrink dynamically as data is added to it. The scope of an EFS is VPC wide.

Let's configure EFS now.

From within the AWS services dashboard select EFS under the storage section.


Since I did not have any previous EFS shares I'm dropped to a getting started sort of screen.


From there click on create file system.

Now we are prompted to select the VPC within which we want to create the EFS. We can also select the availability zone, subnet, IP address and security groups.


Next we specify a name tag for our EFS.


After typing in the name tag click on next step.
Now we are brought to a review page to go through our selections. Since we are satisfied with the selections click on create file system.


And that's it. Our EFS share has been created and we'll be shown the below screen


This gives summary information of the EFS we just created and a link to access mount instructions to mount the file system on an EC2 instance.
Note that the lifecycle state is creating at the moment. It takes a few minutes to complete.

Let's click on the EC2 mount instructions to see how we could mount the EFS.



Now, I'll login to my instance that I created in a previous article within the same VPC and same subnet as the EFS and try to mount the EFS.

[ec2-user@ip-192-168-1-150 ~]$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 12:c0:01:7a:c7:7c brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.150/24 brd 192.168.1.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::10c0:1ff:fe7a:c77c/64 scope link
       valid_lft forever preferred_lft forever
[ec2-user@ip-192-168-1-150 ~]$
[ec2-user@ip-192-168-1-150 ~]$ sudo yum install -y nfs-utils
Loaded plugins: priorities, update-motd, upgrade-helper
amzn-main                                                                                                                                                                                                          | 2.1 kB  00:00:00
amzn-updates                                                                                                                                                                                                       | 2.3 kB  00:00:00
(1/5): amzn-main/latest/group                                                                                                                                                                                      |  35 kB  00:00:00
(2/5): amzn-updates/latest/group                                                                                                                                                                                   |  35 kB  00:00:00
(3/5): amzn-updates/latest/updateinfo                                                                                                                                                                              | 390 kB  00:00:01
(4/5): amzn-updates/latest/primary_db                                                                                                                                                                              | 322 kB  00:00:01
(5/5): amzn-main/latest/primary_db                                                                                                                                                                                 | 3.6 MB  00:00:03
Package 1:nfs-utils-1.3.0-0.21.amzn1.x86_64 already installed and latest version
Nothing to do
[ec2-user@ip-192-168-1-150 ~]$ sudo mkdir /sahil_efs_test
[ec2-user@ip-192-168-1-150 ~]$ ls -ld /sahil_efs_test
drwxr-xr-x 2 root root 4096 Jun 12 09:31 /sahil_efs_test
[ec2-user@ip-192-168-1-150 ~]$ sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 fs-a349c8ea.efs.us-east-1.amazonaws.com:/efs /sahil_efs_test


So, everything seemed fine but I wasn't able to mount the EFS.

Now it took some thinking to come to realize that the security group I applied to this instance didn't actually any incoming NFS traffic from anywhere and also the EFS did not have that security group associated with it.

First I rectified the EFS side. To do so select the file system and under actions select manage file system access.



I then added the name of the security group applied to my instance in the security group section for the EFS and clicked on save.


If we go to the file system section again and view details of our EFS we can see that it's mount target has been updated and the second security group has been added.


With that done, now I added a rule in the security group to allow all incoming traffic from within the same security group.

 

Since the EFS and the instance are in the same security group incoming NFS traffic from the EFS should be allowed to the instance and we should be able to mount the EFS on a directory in our instance.

[ec2-user@ip-192-168-1-150 ~]$ sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 fs-a349c8ea.efs.us-east-1.amazonaws.com:/ /sahil_efs_test
[ec2-user@ip-192-168-1-150 ~]$
[ec2-user@ip-192-168-1-150 ~]$ df -h /sahil_efs_test
Filesystem                                 Size  Used Avail Use% Mounted on
fs-a349c8ea.efs.us-east-1.amazonaws.com:/  8.0E     0  8.0E   0% /sahil_efs_test
[ec2-user@ip-192-168-1-150 ~]$
[ec2-user@ip-192-168-1-150 ~]$ df -k /sahil_efs_test
Filesystem                                       1K-blocks  Used        Available Use% Mounted on
fs-a349c8ea.efs.us-east-1.amazonaws.com:/ 9007199254740992     0 9007199254740992   0% /sahil_efs_test
[ec2-user@ip-192-168-1-150 ~]$
[ec2-user@ip-192-168-1-150 ~]$ df -hTP /sahil_efs_test
Filesystem                                Type  Size  Used Avail Use% Mounted on
fs-a349c8ea.efs.us-east-1.amazonaws.com:/ nfs4  8.0E     0  8.0E   0% /sahil_efs_test

1 comment:

  1. The programming for creating AWS EFS looks quite a complex for me but i will try and understand it as extracting data for me is very essential. Thanks for the program.

    ReplyDelete

Using capture groups in grep in Linux

Introduction Let me start by saying that this article isn't about capture groups in grep per se. What we are going to do here with gr...