I always want to share some of the quick-tip like the one below.
Recently when I was working with Amazon Cloud for one of our client I faced a problem when umount (dismount) a volume (EBS – Elastic Block Storage) (OR) detaching from an EC2 instance. I could not dismount the volume, whenever I try to dismount, it throws “umount: … device is busy“. I was like not sure what is going on, why the device is not dismounting from the machine ???.
Dismounting a particular device forcefully, especially a storage volume can cause a serious damage you will end-up losing some important data. I was not sure what should I do, on the other end, the PRESSURE, I have to dismount the volume and re-create another machine.
After few tries I gave-up and I had to dismount the device forcefully, but luckily none of the files were corrupt, since none of the process were doing any I/O operation on this device but I have no clue why it was behaving different. Below I have explained the root cause of this problem and how to detach a volume from Amazon EC2 instance.
- I detached the volume from Amazon Console.
- Now, login (SSH) into the machine and trying to umount the device.
This is the problem. You should not detach the volume from an instance until and otherwise it is safely removed (dismount) from the machine.
If you are decided to detach or dismount a volume from your machine, you should do:
- Login (SSH) into the machine.
- Make sure your device is not busy means no file operation should be performed when you dismount a device
- Use following commands to make sure the device is not BUSY.
- # lsof | grep ‘/dev/sda’ – “/dev/sda” is my device, change this to match your needs.
- The above command gives you a list of all the process using this particular device.
- Kill all of those process (ex: kill -9 <process_id>)
- Now, run the following command:
- #umount /dev/sda
- If you still not able to dismount the device, then try the following option:
- #umount -l /dev/sda
- -l : Also known as Lazy umount (dismount). Detach the filesystem from the file-system hierarchy now, and cleanup all references to the filesystem as soon as it is not busy anymore
- #umount -f /dev/sda
- -f: Force umount (dismount) in case of an unreachable NFS system
- CAUTION: using these commands can cause data loss for those files opened by any process;
- Now, detach the volume from your Amazon console.
I believe this is not a newbie TIP, soon I am going to write quite a bit of blogs on Amazon Cloud.
Thanks for reading.
Amazon EBS Volume Size Mismatch b/w AWS Console & device attached in Linux (rakeshsankar.wordpress.com)