I am migrating the main file servers at work onto a new storage platform based on GPFS. I am using RHEL 5.6 with the samba3x packages (aka 3.5.4) recompiled to get the vfs_gpfs and tsmsm modules, with a couple of extra patches to vfs_gpfs module to bring it 3.5.8 level. It is running with ctdb against Windows AD 2008 R2 domain controllers with all the idmapping been held in the AD. In order to get robust and frankly usable quotas we are making extensive use of filesets (think directory quotas and you won't go far wrong). Basically every share is in a fileset of it's own including all the users home directories. All the filesets have a quota attached to them. What I would like is to have the disk size and usage reported by windows to be quota limit and usage for the fileset, rather than for the entire file system as at over 100TB it is somewhat misleading. I thought I would be able to use the dfree command option of smb.conf to report the correct information gathered through a script of some description. Unfortunately even with a simple shell script that echos a couple of numbers is owned by root and has permissions 700 does nothing (I still see numbers for the entire file system not the specific share) and I can see nothing in the samba logs even at log level 5. Am I doing something wrong or is this bust? I tried making it work with a plain CentOS 5.6 with straight samba packages on ext3 with similar results. Alternatively would it be possible to add some quota support to vfs_gpfs to make this work? There are plenty of errors of the form [2011/06/07 15:53:15.672182, 3] lib/sysquotas.c:453(sys_get_quota) sys_get_vfs_quota() failed for mntpath[/lifesci] bdev[/dev/lifesci] qtype[2] id[9651]: No such device I am guessing that standard quota calls don't work on GPFS file systems. JAB. -- Jonathan A. Buzzard Email: jonathan (at) buzzard.me.uk Fife, United Kingdom.
On Tue, Jun 07, 2011 at 04:10:43PM +0100, Jonathan Buzzard wrote:> > I am migrating the main file servers at work onto a new storage platform > based on GPFS. I am using RHEL 5.6 with the samba3x packages (aka 3.5.4) > recompiled to get the vfs_gpfs and tsmsm modules, with a couple of extra > patches to vfs_gpfs module to bring it 3.5.8 level. It is running with > ctdb against Windows AD 2008 R2 domain controllers with all the > idmapping been held in the AD. > > In order to get robust and frankly usable quotas we are making extensive > use of filesets (think directory quotas and you won't go far wrong). > Basically every share is in a fileset of it's own including all the > users home directories. All the filesets have a quota attached to them. > > What I would like is to have the disk size and usage reported by windows > to be quota limit and usage for the fileset, rather than for the entire > file system as at over 100TB it is somewhat misleading. > > I thought I would be able to use the dfree command option of smb.conf to > report the correct information gathered through a script of some > description. Unfortunately even with a simple shell script that echos a > couple of numbers is owned by root and has permissions 700 does nothing > (I still see numbers for the entire file system not the specific share) > and I can see nothing in the samba logs even at log level 5.dfree should work fine. I'd persue this avenue.> Am I doing something wrong or is this bust? I tried making it work with > a plain CentOS 5.6 with straight samba packages on ext3 with similar > results. > > Alternatively would it be possible to add some quota support to vfs_gpfs > to make this work? There are plenty of errors of the form > > [2011/06/07 15:53:15.672182, 3] lib/sysquotas.c:453(sys_get_quota) > sys_get_vfs_quota() failed for mntpath[/lifesci] bdev[/dev/lifesci] > qtype[2] id[9651]: No such device > > I am guessing that standard quota calls don't work on GPFS file systems.Probably true. If you know how GPFS reports quotas we can add the code for a later release. We have quota hooks in the Samba VFS, so we could add this to the gpfs vfs module. Jeremy.
Jonathan wrote on 06/07/2011 05:10:43 PM:> In order to get robust and frankly usable quotas we are making extensive > use of filesets (think directory quotas and you won't go far wrong). > Basically every share is in a fileset of it's own including all the > users home directories. All the filesets have a quota attached to them. > > What I would like is to have the disk size and usage reported by windows > to be quota limit and usage for the fileset, rather than for the entire > file system as at over 100TB it is somewhat misleading.You can use mmchfs <filesystem> --filesetdf to get what you are looking for. Diskfree lookups will then see the remaining quota in the fileset instead of the free space in the whole filesystem. You'll need GPFS 3.3 or later for this flag to be available. Regards, Christian