Power Outage Corrupted XFS Filesystem | How I Fixed It

Dell Inspiron 20 3048-07-Calendar

This past Monday, 27 May 2019, there was a somewhat severe storm that rolled through Southwestern Michigan that had a disruption on power. I have numerous computers in the house, most of which run some variation of openSUSE. Most of the computers are also battery backed in some form except for one, my Kitchen Command Center. In many ways, I think it is rather crazy computers don’t now have battery backups by default. Since I didn’t take the time and care to have a UPS (Uninterruptible Power Supply) on this computer, it lost power and so my troubles began.

Post Storm

After the storm was cleared from the skies and the likelihood of another power outage had diminished I felt it was safe to power my devices back on. Upon doing so, the machine booted up like it normally would but logging in and all the applications were dreadfully slow. I must emphasis dreadfully in the slowness of the system, I could see that the disk was thrashing but the RAM was hardly being used. Looking at the System Monitor, my I/O was taking up all the CPU bandwidth. This was most certainly not the normal behavior of this machine and I was becoming a sad “Geeko.” RAM usage was less than 2 GiB and Plasma Desktop kept hanging, a behavior with which I am most certainly not familiar. I was starting to worry that there may have been hardware damage.

After doing a little digging, I was able to determine that it was related to a corrupted file system and in my usage of the computer, my estimation was that it was the /home directory partition and not the root directory. When I looked at the System Activity, whatever application I was trying to use had “disk sleep” next to it in the table. My first course of action was to do a file system repair.

The Fix

I rebooted the machine and instead of logging in to the desktop environment, I dropped down to a terminal (Ctrl + Alt + F2), logged in as root.

I unmounted /home, which, in my case is located at /dev/sda4

umount /dev/sda4

Since the terminal didn’t give me any confirmation the drive was unmounted, I checked

df -h

Looking through the list (You can omit the -h). I saw that there was indeed nothing mounted at /home so I was able to conduct the repair.

xfs_repair /dev/sda4

After several minutes. The process completed and seemingly completed without any errors. I rebooted the system and crossed my fingers.

Post Repair

Seemingly everything is back to normal. Whatever was causing the “Disk Sleep” is not happening anymore. I performed another update on the machine,

sudo zypper dup

rebooted it and it is continuing to function just as it had before. I have not lost any data on the computer and I am using like it all never even happened. I don’t know the exact cause and depth of the corruption but I am just glad to be back to normal.

Final Thoughts

I have had to forcibly shut down systems with XFS before and this is the first time I have had to do a file system repair. I could see that someone without technical expertise could just think their computer was broken and take more intrusive actions. I am also not sure if there was some sort of file system integrity verification that didn’t happen that should have automatically checked and repaired the file system that has normally done so. Regardless, the fix was relatively straight forward and the computer is back to normal. Furthermore, it might also behoove me to gift the machine with a UPS.

After losing a few hours of use out of the computer, I was able to learn another tool in my open source / Linux toolbox. The storm, although inconvenient, has given me further confidence in the technology I have chosen.

Further Reading

https://linux.die.net/man/8/xfs_repair
Dell Inspiron 20 3048 All-In-One Desktop
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.