Home > Developer, Hardware, Kernel / Internals > Maintaining your XFS with XFS Filesystem Reorganiser xfs_fsr to defrag

Maintaining your XFS with XFS Filesystem Reorganiser xfs_fsr to defrag

January 25th, 2009 Leave a comment Go to comments

File Systems are a hairy topic, on Windows you should be using NTFS (the days of FAT are long gone!) but on Linux, BSD and *Solaris we still have a wide variety to pick and choose depending on our needs. I’ve always been a JFS and XFS fan (previously ReiserFS) until Btrfs goes mainstream (which is one thing to hangout for in Linux Kernel 2.6.29!) and often I’d have a mixture of all three. Our main server at home – affectionately dubbed Zeus, after our lovable Australian Customs puppy Zeus, uses XFS, JFS and Ext3.

JFS to manage the home directories and core file system, ReiserFS for the temp folder and XFS for the heavy file shares – which span multiple terrabytes of files over a LVM (with each file being 1-2Gb in size). The reasoning behind opting for XFS over another file system for the file server was that XFS performs incredibly well under heavy load and scales well when you know the files are big (over 500Mb). Overall I’ve always felt that XFS does provide consistent performance and scalabilty in comparison to the others – but you may think otherwise.

Unfortunately, XFS – whilst quite an excellent file system for managing large files, it seems, suffers from fragmentation over time (especially true if you use your file system for DVR – eg, a Myth backend host) or if the disk gets close to filling up. Luckily there are two utilities that XFS has to manage this fragmentation.

  • xfs_db – XFS Debug Information
    Used to examine an XFS filesystem for problems or gather information about the XFS file system.
  • xfs_fsr – File System Organiser
    Improves the organisation of mounted file systems. The reorganisation algorithm operates on one file at a time, compacting or otherwise improving the layout of the file extents (contiguous blocks of file data).

In Debian/Ubuntu (and derivatives) these two utilities are found in the package xfsdump. Using these two utilities we can workout the health of the file system (xfs_db) and hopefully tune/optimise it (xfs_fsr). I took the plunge last night and optimised Zeus’s main file storage partition:

Filesystem            Size Used Avail Use% Mounted on
/dev/sdf7              40G  3.5G   37G   9% /
varrun               1014M  4.5M 1010M   1% /var/run
varlock              1014M  8.0K 1014M   1% /var/lock
udev                 1014M  112K 1014M   1% /dev
devshm               1014M     0 1014M   0% /dev/shm
lrm                  1014M   34M  980M   4% /lib/modules/2.6.22-15-generic/volatile
/dev/sdf6            1023M   38M  986M   4% /boot
/dev/sdf10            235G  173G   63G  74% /home
/dev/sdf9              10G  544K   10G   1% /opt
/dev/sdf8              10G  2.7G  7.4G  27% /var
/dev/mapper/Storage
                      2.3T  1.9T  408G  83% /media/LVM/Storage
/dev/sde1             466G  396G   71G  85% /media/Backups

As you can see, the LVM “Storage” mount has just under 20% free space and the non-LVM partition for “Backups” has 15% free space. Both these are XFS volumes, to find the health of the two use the xfs_db command to gather some information.

$ sudo  xfs_db -c frag -r /dev/mapper/Storage
$ sudo  xfs_db -c frag -r /dev/sde1

Here we’re asking xfs_db to open the file system in a readonly mode (-r) passing in a command (-c)  to get the file fragementation data (frag) for the device (/dev/*). When we use the frag command, it returns information only pertaining to the file data in the filesystem as opposed to the fragmentation of freespace (which we can guage with passing the freesp command). The output of the commands appear below for Zeus.

thushan@ZEUS:~$ sudo  xfs_db -c frag -r /dev/sde1
actual 189356, ideal 148090, fragmentation factor 21.79%

thushan@ZEUS:~$ sudo  xfs_db -c frag -r /dev/mapper/Storage
actual 406056, ideal 21584, fragmentation factor 94.68%

Wow! The LVM partition (which spans 4 drives) has around 95% fragementation! Yikes!!! The parition has quite a few Virtual Machine images, various large files (DV Captures etc). The “Backup” (sde1) on the other hand isnt as badly fragmented.

So right now we’ve found our problem and its time to fix it. First thing to do – and realise that we can fix this on a live running system – is to try and find a time where the partition will be used very little (like overnight) so you let its do its thing without unnecessary burden. Then lets make use of the File System Organiser utility (xfs_fsr) and ask it to reorganise our parition to the best of its ability.

$ sudo xfs_fsr -t 25200 /dev/mapper/Storage -v
$ sudo xfs_fsr -t 25200 /dev/sde1 -v

Now this is much simpler, the xfs_fsr utility is being told to reorganise /dev/* with a timeout (-t) of 7hrs  (60 * 60 * 7 = 25200) which is specified in seconds. Because I like to see how much is done I also specified the verbose output option (-v). Let it do its thing and hopefully when you return you will have the last bit of output showing the extents before, how many after and the inode, something like this:

extents before:5 after:1 DONE ino=4209066103
ino=4209066107
extents before:5 after:1 DONE ino=4209066107
ino=4209066101
extents before:4 after:1 DONE ino=4209066101
ino=4209066091
extents before:3 after:1 DONE ino=4209066091
ino=4209066093
extents before:3 after:1 DONE ino=4209066093
ino=4209066105
extents before:2 after:1 DONE ino=4209066105
ino=4209066143
extents before:27 after:1 DONE ino=4209066143

Now its time to go back and check how well the file system reorganising was:

$ sudo  xfs_db -c frag -r /dev/mapper/Storage

And the results?

thushan@ZEUS:~$ sudo  xfs_db -c frag -r /dev/mapper/Storage
actual 21652, ideal 21584, fragmentation factor 0.31%

Lovely! What a difference and you’ll notice the improvement immediately if you start moving or transfering files around.

Ideally, you may want to setup a cron task to let this process run (maybe with a lower timeout) overnight or when theres low-load. Whats great about the xfs_fsr utility is that its smart enough to remember where it finished up last time and continue from there. Its a shame Ubuntu doesnt do this already.

{lang: 'en-GB'}
Share

Related posts:

  1. Linux 2.6.28 released for all the good l33tle boys and girls!
  2. OpenSolaris 2008.11 out the door!!!
  3. Sun ushers in VirtualBox 2.1 with cool new features!
  4. Breaking News: BD+ Broken
  1. January 27th, 2009 at 16:14 | #1

    hey thanks for the info i mine was 98% fragmented and now after 8 ours its down to less than 1%, please keep posting about linux stuff

  2. wilee
    September 10th, 2009 at 08:23 | #2

    ubuntu myth has an option in the gui myth controller to defrag xfs , mine was at %30 when I started :)

  3. Seth
    September 29th, 2009 at 09:01 | #3

    This was excellent information indeed. I found out my VM volume was dramatically fragmented (49%) and getting very slow read performance (stretches of 10Mb/s in sequential _file_ reads, as opposed to ~70Mb/s when unfragment, i.e. sequential _disk_ reads).

    It turned out I had to feed XFS some more extents for it to even start reorganizing, but then it went smooth! I’ll run xfs_fsr against the boot profile (readahead) on my ubuntu box to see what happens!

  4. Jeffrey Hundstad
    June 8th, 2011 at 00:12 | #4

    Here’s a response to WHY this occurs:

    http://old.nabble.com/Defragging-XFS-File-Systems-to31788982.html#a31789236

    Re: Defragging XFS File Systems

    by Stewart Smith-2 Jun 07, 2011; 12:48am

    On Mon, 6 Jun 2011 22:52:48 -0500, Kenneth Emerson wrote:
    > I hadn’t given much thought to fragmentation of my TV recordings volume
    > (XFS) until reading through some MythTV-users threads recently that
    > mentioned how fragmented an XFS file system could become. After running
    > xfs_db, I found out that my fs appeared to be quite bad:

    MythTV can end up with fragmentation on XFS due to an fsync() call that
    attempts to work raound limitations in ext3.

    Workarounds include:
    - allocsize mount parameter
    - patch mythtv source not to fsync (you could easily write a patch that
    only did fsync if not xfs… I’ve been meaning to do this for
    years… not enough hours in day).
    - run mythbackend with libeatmydata, thus disabling the fsync


    Stewart Smith

  5. KevinLS
    November 17th, 2011 at 13:53 | #5

    Looks like this article is a couple years old; but I’m sure glad I found it.

    Very helpful!

    Thanks!

  6. customized url shortener
    September 26th, 2012 at 05:46 | #6

    Hi everybody, here every person is sharing these
    kinds of knowledge, so it’s fastidious to read this website, and I used to pay a visit this website everyday.

  1. February 23rd, 2009 at 22:21 | #1
  2. October 18th, 2009 at 11:46 | #2
  3. November 13th, 2009 at 07:07 | #3