Amazon EBS VOLUME – Attach/Detach – Mount/UMount – device is busy [SOLUTION]


Unix filesystem
Image via Wikipedia

I always want to share some of the quick-tip like the one below.

Recently when I was working with Amazon Cloud for one of our client I faced a problem when umount (dismount) a volume (EBSElastic Block Storage) (OR) detaching from an EC2 instance. I could not dismount the volume, whenever I try to dismount, it throws “umount: … device is busy“. I was like not sure what is going on, why the device is not dismounting from the machine ???.

Dismounting a particular device forcefully, especially a storage volume can cause a serious damage you will end-up losing some important data. I was not sure what should I do, on the other end, the PRESSURE, I have to dismount the volume and re-create another machine.

After few tries I gave-up and I had to dismount the device forcefully, but luckily none of the files were corrupt, since none of the process were doing any I/O operation on this device but I have no clue why it was behaving different. Below I have explained the root cause of this problem and how to detach a volume from Amazon EC2 instance.

Root Cause:

  1. I detached the volume from Amazon Console.
  2. Now, login (SSH) into the machine and trying to umount the device.

This is the problem. You should not detach the volume from an instance until and otherwise it is safely removed (dismount) from the machine.

Solution:

If you are decided to detach or dismount a volume from your machine, you should do:

  1. Login (SSH) into the machine.
  2. Make sure your device is not busy means no file operation should be performed when you dismount a device
    1. Use following commands to make sure the device is not BUSY.
      1. # lsof | grep ‘/dev/sda’ – “/dev/sda” is my device, change this to match your needs.
    2. The above command gives you a list of all the process using this particular device.
    3. Kill all of those process (ex: kill -9 <process_id>)
    4. Now, run the following command:
      1. #umount /dev/sda
    5. If you still not able to dismount the device, then try the following option:
      1. #umount -l /dev/sda
        1. -l : Also known as Lazy umount (dismount). Detach the filesystem from the file-system hierarchy now, and cleanup all references to the filesystem as soon as it is not busy anymore
      2. #umount -f /dev/sda
        1. -f: Force umount (dismount) in case of an unreachable NFS system
      3. CAUTION: using these commands can cause data loss for those files opened by any process;
  3. Now, detach the volume from your Amazon console.

I believe this is not a newbie TIP, soon I am going to write quite a bit of blogs on Amazon Cloud.

Thanks for reading.

Related Articles:

    1. Amazon EBS Volume Size Mismatch b/w AWS Console & device attached in Linux (rakeshsankar.wordpress.com)

Amazon EBS VOLUME – Attach/Detach – Mount/UMount – device is busy [SOLUTION]

2 thoughts on “Amazon EBS VOLUME – Attach/Detach – Mount/UMount – device is busy [SOLUTION]

  1. […] Amazon EBS VOLUME – Attach/Detach – Mount/UMount – device is busy [SOLUTION] LD_AddCustomAttr("AdOpt", "1"); LD_AddCustomAttr("Origin", "other"); LD_AddCustomAttr("theme_bg", "f3f3f3"); LD_AddCustomAttr("theme_text", "888888"); LD_AddCustomAttr("theme_link", "b91313"); LD_AddCustomAttr("theme_border", "d2d2d2"); LD_AddCustomAttr("theme_url", "b91313"); LD_AddCustomAttr("LangId", "1"); LD_AddCustomAttr("Autotag", "technology"); LD_AddCustomAttr("Tag", "amazon-aws"); LD_AddCustomAttr("Tag", "linux"); LD_AddCustomAttr("Tag", "ssh"); LD_AddCustomAttr("Tag", "amazon-ebs"); LD_AddCustomAttr("Tag", "amazon-ec2"); LD_AddCustomAttr("Tag", "debian-2"); LD_AddCustomAttr("Tag", "ssh-2"); LD_AddSlot("wpcom_below_post"); LD_GetBids(); SHAREsome:Like this:LikeBe the first to like this post. Leave a Comment by Rakesh Sankar on December 13, 2011  •  Permalink Posted in Amazon AWS, Linux, SSH Tagged Amazon EBS, Amazon EC2, debian, ssh […]

  2. tim-in-oakton says:

    I would not recommend initially using a kill -9 to reap the processes accessing the mounted device as “-9” kind of pulls the rug out from under the process and doesn’t allow the process to gracefully terminate. Using kill -9 can lead to referential ingrity problems, incomplete writes and all sorts of corner cases that will lead to unhappiness. Use a lesser kill by default will make it much easier for the processes to gracefully terminate.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s