Ran into a situation that I thought had been corrected in the 2.0.x time
frame.
[root at X03 ~]# ls -alF
/data/brick-sdc2/glusterfs/dht/opra/20110502/options_20110502_opra_ch_015_.dat.zip.stats
/data/brick-sdd2/glusterfs/dht/opra/20110502/options_20110502_opra_ch_015_.dat.zip
---------T 1 root root 0 Jul 19 18:08
/data/brick-sdc2/glusterfs/dht/opra/20110502/options_20110502_opra_ch_015_.dat.zip.stats
---------T 1 root root 0 Jul 21 10:48
/data/brick-sdd2/glusterfs/dht/opra/20110502/options_20110502_opra_ch_015_.dat.zip
and the real files
[root at X02 ~]# ls -alF
/data/glusterfs/dht/opra/20110502/options_20110502_opra_ch_015_*
-rw-r--r-- 1 oracle oinstall 984227914 Jul 21 11:05
/data/glusterfs/dht/opra/20110502/options_20110502_opra_ch_015_.dat.zip
-rw-r--r-- 1 root root 17028780 Jul 20 13:00
/data/glusterfs/dht/opra/20110502/options_20110502_opra_ch_015_.dat.zip.stats
Note the permissions on the first as compared to the second. This came
from a rebalance operation with 3.1.5.
Anyone have that handy
find -perm XXXX -exec rm {}
command handy so we can scan for and remove the ghost files? I can
search for it in my old emails, just figured I'd ask.
--
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics, Inc.
email: landman at scalableinformatics.com
web : http://scalableinformatics.com
http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax : +1 866 888 3112
cell : +1 734 612 4615
Hi Joe.
Before I answer - can you share your volume info dump?
My understanding is that those "link" files are valid if you have my
kind of config - which is distributed-replicate across 2 mirrored pairs of
backed servers. If the request for a file come into the pair that do not have
that file physically on their storage, the "link" file is created to
point to the actual location on the other mirror. At least ... that is what I
think it's supposed to do ...
James Burnash
Unix Engineer
Knight Capital Group
-----Original Message-----
From: gluster-users-bounces at gluster.org [mailto:gluster-users-bounces at
gluster.org] On Behalf Of Joe Landman
Sent: Thursday, July 21, 2011 11:11 AM
To: gluster-users
Subject: [Gluster-users] Anyone have that find command handy
Ran into a situation that I thought had been corrected in the 2.0.x time frame.
[root at X03 ~]# ls -alF
/data/brick-sdc2/glusterfs/dht/opra/20110502/options_20110502_opra_ch_015_.dat.zip.stats
/data/brick-sdd2/glusterfs/dht/opra/20110502/options_20110502_opra_ch_015_.dat.zip
---------T 1 root root 0 Jul 19 18:08
/data/brick-sdc2/glusterfs/dht/opra/20110502/options_20110502_opra_ch_015_.dat.zip.stats
---------T 1 root root 0 Jul 21 10:48
/data/brick-sdd2/glusterfs/dht/opra/20110502/options_20110502_opra_ch_015_.dat.zip
and the real files
[root at X02 ~]# ls -alF
/data/glusterfs/dht/opra/20110502/options_20110502_opra_ch_015_*
-rw-r--r-- 1 oracle oinstall 984227914 Jul 21 11:05
/data/glusterfs/dht/opra/20110502/options_20110502_opra_ch_015_.dat.zip
-rw-r--r-- 1 root root 17028780 Jul 20 13:00
/data/glusterfs/dht/opra/20110502/options_20110502_opra_ch_015_.dat.zip.stats
Note the permissions on the first as compared to the second. This came from a
rebalance operation with 3.1.5.
Anyone have that handy
find -perm XXXX -exec rm {}
command handy so we can scan for and remove the ghost files? I can search for
it in my old emails, just figured I'd ask.
--
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics, Inc.
email: landman at scalableinformatics.com
web : http://scalableinformatics.com
http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax : +1 866 888 3112
cell : +1 734 612 4615
_______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
DISCLAIMER:
This e-mail, and any attachments thereto, is intended only for use by the
addressee(s) named herein and may contain legally privileged and/or confidential
information. If you are not the intended recipient of this e-mail, you are
hereby notified that any dissemination, distribution or copying of this e-mail,
and any attachments thereto, is strictly prohibited. If you have received this
in error, please immediately notify me and permanently delete the original and
any copy of any e-mail and any printout thereof. E-mail transmission cannot be
guaranteed to be secure or error-free. The sender therefore does not accept
liability for any errors or omissions in the contents of this message which
arise as a result of e-mail transmission.
NOTICE REGARDING PRIVACY AND CONFIDENTIALITY Knight Capital Group may, at its
discretion, monitor and review the content of all e-mail communications.
http://www.knight.com
*> *Luis Cerezo* lec at luiscerezo.org > <mailto:gluster-users%40gluster.org?Subject=Re%3A%20%5BGluster-users%5D%20Anyone%20have%20that%20find%20command%20handy&In-Reply-To=%3C3F7AF906-46B9-4166-8F6D-5400FB0B7EE9%40luiscerezo.org%3E> > /Thu Jul 21 17:19:45 PDT 2011/ > ------------------------------------------------------------------------ > i would also add a -size 0. > > -luis >> On Jul 21, 2011, at 9:30 AM, Joe Landman wrote: >> >> >/ On 07/21/2011 11:21 AM, Joe Landman wrote: >> />>/ On 07/21/2011 11:17 AM, Burnash, James wrote: >> />>>/ Hi Joe. >> />>>/ >> />/ >> />/ Found it: >> />/ >> />/ find $local -type f -perm +01000 -exec rm -v '{}' \; >> />/ >> />/ for $local >> />/ >> />/ / >*I too would strongly suggest this, because I have been seeing real, non-zero size files with "---------T" permissions since upgrading to 3.2.0. These files are visible at the mount point and usually (although not always) have the correct ownership (i.e. non root). In each case there is always a different version of the file on another pair of bricks, which is different in size and almost always older than the 'T' file (as I call them), but I did see one case where the alternative version was newer than the 'T' file. I have only seen a few dozen of these so far, but what worries me is that GlusterFS could be deleting them as if they are "stale link files", a message I often see in the client logs. I have no idea which of the two alternative versions of these files are correct - the 'T' files or their alternatives - possibly both. They might both be valid output from two successive model runs for example. Only the owners of these files can say which versions are correct; one user has agreed to take a look at some examples that I saved but hasn't had time to do it yet. Deleting a 'T' file (or moving it to another location as I always do) results in the alternative version appearing at the mount point. The same thing happens after removing the trusted.gfid xattr from the 'T' file, as is done to files with GFID errors by Vikas Gorur's new GFID mismatch fixing tools I believe. Therefore there is a real danger of 'T' files being deleted, either by "find ... -execrm{} \;" or by GlusterFS itself, resulting in the loss of important data. What certainly does happen is that some of the 'T' files become hidden from the mount point (possibly as a result of GFID matcher fixer scripts), and even if they are not actually deleted it can be just as disruptive for the files' owners. I sincerely hope that the GFID bug fixes in 3.2.2 will stop this from occurring, but I don't know if the 'T' file problem is related to GFID mismatches and it is too early to tell whether or not the problem has been solved by upgrading to 3.2.2. I am going to monitor the situation by doing "find . -perm 1000 -size +0b -exec /bin/ls -l {} \;" in the mount points of all the volumes, and on the back-end filesystems, as often as I can. -Dan. / / -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20110723/4c9ada70/attachment.html>