I have a problem with file mod times using AFR. this is really bad: server1# ls -al vfc017.jpg -rw-r--r-- 1 user1 group1 47660 2008-07-10 14:06 vfc017.jpg server1# ls -alu vfc017.jpg -rw-r--r-- 1 user1 group1 47660 2008-07-10 14:06 vfc017.jpg server2# ls -al vfc017.jpg -rw-r--r-- 1 user1 group1 47660 2008-12-23 09:25 vfc017.jpg server2# ls -alu vfc017.jpg -rw-r--r-- 1 user1 group1 47660 2008-12-29 15:55 vfc017.jpg so, there are programs and such which look for files modified after a certain date, and does stuff to them. if this is a web app, for example, and it hits the server with the newer date (the date AFR copied the file I presume?) then it thinks every single thing is new and does it''s processing. Shouldn''t AFR set the file mod and create times to that of the original source file? However, once it''s in this state, I''m not sure how to fix it in an automated way, but hopefully there will be a patch so that this doesn''t happen in the future. Keith
I have a problem with file mod times using AFR. this is really bad: server1# ls -al vfc017.jpg -rw-r--r-- 1 user1 group1 47660 2008-07-10 14:06 vfc017.jpg server1# ls -alu vfc017.jpg -rw-r--r-- 1 user1 group1 47660 2008-07-10 14:06 vfc017.jpg server2# ls -al vfc017.jpg -rw-r--r-- 1 user1 group1 47660 2008-12-23 09:25 vfc017.jpg server2# ls -alu vfc017.jpg -rw-r--r-- 1 user1 group1 47660 2008-12-29 15:55 vfc017.jpg so, there are programs and such which look for files modified after a certain date, and does stuff to them. if this is a web app, for example, and it hits the server with the newer date (the date AFR copied the file I presume?) then it thinks every single thing is new and does it''s processing. Shouldn''t AFR set the file mod and create times to that of the original source file? However, once it''s in this state, I''m not sure how to fix it in an automated way, but hopefully there will be a patch so that this doesn''t happen in the future. Keith
I have a problem with file mod times using AFR. this is really bad: server1# ls -al vfc017.jpg -rw-r--r-- 1 user1 group1 47660 2008-07-10 14:06 vfc017.jpg server1# ls -alu vfc017.jpg -rw-r--r-- 1 user1 group1 47660 2008-07-10 14:06 vfc017.jpg server2# ls -al vfc017.jpg -rw-r--r-- 1 user1 group1 47660 2008-12-23 09:25 vfc017.jpg server2# ls -alu vfc017.jpg -rw-r--r-- 1 user1 group1 47660 2008-12-29 15:55 vfc017.jpg so, there are programs and such which look for files modified after a certain date, and does stuff to them. if this is a web app, for example, and it hits the server with the newer date (the date AFR copied the file I presume?) then it thinks every single thing is new and does it''s processing. Shouldn''t AFR set the file mod and create times to that of the original source file? However, once it''s in this state, I''m not sure how to fix it in an automated way, but hopefully there will be a patch so that this doesn''t happen in the future. Keith
Keith, It was fixed in patch-804. Krishna On Tue, Dec 30, 2008 at 6:07 AM, Keith Freedman <freedman at freeformit.com> wrote:> I have a problem with file mod times using AFR. > > this is really bad: > server1# ls -al vfc017.jpg > -rw-r--r-- 1 user1 group1 47660 2008-07-10 14:06 vfc017.jpg > server1# ls -alu vfc017.jpg > -rw-r--r-- 1 user1 group1 47660 2008-07-10 14:06 vfc017.jpg > > server2# ls -al vfc017.jpg > -rw-r--r-- 1 user1 group1 47660 2008-12-23 09:25 vfc017.jpg > server2# ls -alu vfc017.jpg > -rw-r--r-- 1 user1 group1 47660 2008-12-29 15:55 vfc017.jpg > > so, there are programs and such which look for files modified after a > certain date, and does stuff to them. > if this is a web app, for example, and it hits the server with the > newer date (the date AFR copied the file I presume?) then it thinks > every single thing is new and does it's processing. > > Shouldn't AFR set the file mod and create times to that of the > original source file? > > However, once it's in this state, I'm not sure how to fix it in an > automated way, but hopefully there will be a patch so that this > doesn't happen in the future. > > Keith > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users >
On Tue, Dec 30, 2008 at 1:07 PM, Keith Freedman <freedman at freeformit.com> wrote:> At 11:00 PM 12/29/2008, Krishna Srinivas wrote: >> >> Keith, >> >> It was fixed in the sense, AFR will return stat of the same subvol >> everytime, before it used to return stat from one of the subvols. But >> still if the subvol-server clock is different than that of client, the >> time stamp will be of the server and not client. > > so, in the case where they both differ, how will we know which time will be > associated with a file?The first subvol mentioned in the list is considered, if it is down, then the next is considered.> and will it fix the on-disk stamp or just ignore it provided the other > subvol is online?It will not fix the time stamp if they differ.> > ideally, it will pick one, and make sure all the others are 'reset' to that > so that if the preferred vol goes offline, timestamps don't suddenly get > weird.True, but ideally all servers and clients should be time synced using ntp. For example if clients are not in sync, one of them might see time stamp of the future.> > as for the server times differing. I suppose it's possible, if so it's by > microseconds as they're both synced hourly to the universal clock via ntpd. > > >> Krishna >> >> On Tue, Dec 30, 2008 at 12:22 PM, Krishna Srinivas >> <krishna at zresearch.com> wrote: >> > Keith, >> > It was fixed in patch-804. >> > Krishna >> > >> > On Tue, Dec 30, 2008 at 6:07 AM, Keith Freedman >> > <freedman at freeformit.com> wrote: >> >> I have a problem with file mod times using AFR. >> >> >> >> this is really bad: >> >> server1# ls -al vfc017.jpg >> >> -rw-r--r-- 1 user1 group1 47660 2008-07-10 14:06 vfc017.jpg >> >> server1# ls -alu vfc017.jpg >> >> -rw-r--r-- 1 user1 group1 47660 2008-07-10 14:06 vfc017.jpg >> >> >> >> server2# ls -al vfc017.jpg >> >> -rw-r--r-- 1 user1 group1 47660 2008-12-23 09:25 vfc017.jpg >> >> server2# ls -alu vfc017.jpg >> >> -rw-r--r-- 1 user1 group1 47660 2008-12-29 15:55 vfc017.jpg >> >> >> >> so, there are programs and such which look for files modified after a >> >> certain date, and does stuff to them. >> >> if this is a web app, for example, and it hits the server with the >> >> newer date (the date AFR copied the file I presume?) then it thinks >> >> every single thing is new and does it's processing. >> >> >> >> Shouldn't AFR set the file mod and create times to that of the >> >> original source file? >> >> >> >> However, once it's in this state, I'm not sure how to fix it in an >> >> automated way, but hopefully there will be a patch so that this >> >> doesn't happen in the future. >> >> >> >> Keith >> >> >> >> >> >> _______________________________________________ >> >> Gluster-users mailing list >> >> Gluster-users at gluster.org >> >> http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users >> >> >> > > >
> > so, in the case where they both differ, how will we know which time will be > > associated with a file? > >The first subvol mentioned in the list is considered, if it is down, >then the next is considered.ok, so I made sure that my subvolume list on the servers is in the same order. I had always listed the local volume first and the remote one second, but it sounds like this would lead to each node using it''s own timestamps or do all the servers in the AFR elect a preferred ''first'' subvol?> > and will it fix the on-disk stamp or just ignore it provided the other > > subvol is online? > >It will not fix the time stamp if they differ.how about new files being replicated.. they''ll get the timestamp of the server from which the source file is available? in other words.. if I add a new empty volume, and AFR copies everything to it... what dates will be on the local files?> > ideally, it will pick one, and make sure all the others are ''reset'' to that > > so that if the preferred vol goes offline, timestamps don''t suddenly get > > weird. > >True, but ideally all servers and clients should be time synced using >ntp. For example if clients >are not in sync, one of them might see time stamp of the future.mine are using ntp, so, again, if they differ,it''s by microseconds. Keith
> > so, in the case where they both differ, how will we know which time will be > > associated with a file? > >The first subvol mentioned in the list is considered, if it is down, >then the next is considered.ok, so I made sure that my subvolume list on the servers is in the same order. I had always listed the local volume first and the remote one second, but it sounds like this would lead to each node using it''s own timestamps or do all the servers in the AFR elect a preferred ''first'' subvol?> > and will it fix the on-disk stamp or just ignore it provided the other > > subvol is online? > >It will not fix the time stamp if they differ.how about new files being replicated.. they''ll get the timestamp of the server from which the source file is available? in other words.. if I add a new empty volume, and AFR copies everything to it... what dates will be on the local files?> > ideally, it will pick one, and make sure all the others are ''reset'' to that > > so that if the preferred vol goes offline, timestamps don''t suddenly get > > weird. > >True, but ideally all servers and clients should be time synced using >ntp. For example if clients >are not in sync, one of them might see time stamp of the future.mine are using ntp, so, again, if they differ,it''s by microseconds. Keith
> > so, in the case where they both differ, how will we know which time will be > > associated with a file? > >The first subvol mentioned in the list is considered, if it is down, >then the next is considered.ok, so I made sure that my subvolume list on the servers is in the same order. I had always listed the local volume first and the remote one second, but it sounds like this would lead to each node using it''s own timestamps or do all the servers in the AFR elect a preferred ''first'' subvol?> > and will it fix the on-disk stamp or just ignore it provided the other > > subvol is online? > >It will not fix the time stamp if they differ.how about new files being replicated.. they''ll get the timestamp of the server from which the source file is available? in other words.. if I add a new empty volume, and AFR copies everything to it... what dates will be on the local files?> > ideally, it will pick one, and make sure all the others are ''reset'' to that > > so that if the preferred vol goes offline, timestamps don''t suddenly get > > weird. > >True, but ideally all servers and clients should be time synced using >ntp. For example if clients >are not in sync, one of them might see time stamp of the future.mine are using ntp, so, again, if they differ,it''s by microseconds. Keith
On Tue, Dec 30, 2008 at 12:29 PM, Keith Freedman <freedman at freeformit.com>wrote:> > > > so, in the case where they both differ, how will we know which time > will be > > > associated with a file? > > > >The first subvol mentioned in the list is considered, if it is down, > >then the next is considered. > > ok, so I made sure that my subvolume list on the servers is in the same > order. > I had always listed the local volume first and the remote one second, > but it sounds like this would lead to each node using it's own > timestamps or do all the servers in the AFR elect a preferred 'first' > subvol? > > > > and will it fix the on-disk stamp or just ignore it provided the other > > > subvol is online? > > > >It will not fix the time stamp if they differ. > > how about new files being replicated.. they'll get the timestamp of > the server from which the source file is available? > > in other words.. if I add a new empty volume, and AFR copies > everything to it... what dates will be on the local files?During self-heal afr sets the timestamps (atime and mtime) to that of the file on the source node.> > > > > ideally, it will pick one, and make sure all the others are 'reset' to > that > > > so that if the preferred vol goes offline, timestamps don't suddenly > get > > > weird. > > > >True, but ideally all servers and clients should be time synced using > >ntp. For example if clients > >are not in sync, one of them might see time stamp of the future. > > mine are using ntp, so, again, if they differ,it's by microseconds. > > Keith > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users >-- Raghavendra G -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20081231/826110ca/attachment.html>
At 09:39 PM 12/30/2008, Raghavendra G wrote:>On Tue, Dec 30, 2008 at 12:29 PM, Keith Freedman ><<mailto:freedman at freeformit.com>freedman at freeformit.com> wrote: > > > > so, in the case where they both differ, how will we know which > time will be > > > associated with a file? > > > >The first subvol mentioned in the list is considered, if it is down, > >then the next is considered. > >ok, so I made sure that my subvolume list on the servers is in the same order. >I had always listed the local volume first and the remote one second, >but it sounds like this would lead to each node using it''s own >timestamps or do all the servers in the AFR elect a preferred ''first'' subvol? > > > > and will it fix the on-disk stamp or just ignore it provided the other > > > subvol is online? > > > >It will not fix the time stamp if they differ. > >how about new files being replicated.. they''ll get the timestamp of >the server from which the source file is available? > >in other words.. if I add a new empty volume, and AFR copies >everything to it... what dates will be on the local files? > > >During self-heal afr sets the timestamps (atime and mtime) to that >of the file on the source node.this works fine. to solve my problem, I created a script which gets the file times for each file and creates a set of "touch -d " commands to set the dates on the other server. that was preferable to deleting the directory tree and having things auto-healed since it''s roughly 12 GB''s And, it seems it''s not something that would be a problem in the future. Keith
At 09:39 PM 12/30/2008, Raghavendra G wrote:>On Tue, Dec 30, 2008 at 12:29 PM, Keith Freedman ><<mailto:freedman at freeformit.com>freedman at freeformit.com> wrote: > > > > so, in the case where they both differ, how will we know which > time will be > > > associated with a file? > > > >The first subvol mentioned in the list is considered, if it is down, > >then the next is considered. > >ok, so I made sure that my subvolume list on the servers is in the same order. >I had always listed the local volume first and the remote one second, >but it sounds like this would lead to each node using it''s own >timestamps or do all the servers in the AFR elect a preferred ''first'' subvol? > > > > and will it fix the on-disk stamp or just ignore it provided the other > > > subvol is online? > > > >It will not fix the time stamp if they differ. > >how about new files being replicated.. they''ll get the timestamp of >the server from which the source file is available? > >in other words.. if I add a new empty volume, and AFR copies >everything to it... what dates will be on the local files? > > >During self-heal afr sets the timestamps (atime and mtime) to that >of the file on the source node.this works fine. to solve my problem, I created a script which gets the file times for each file and creates a set of "touch -d " commands to set the dates on the other server. that was preferable to deleting the directory tree and having things auto-healed since it''s roughly 12 GB''s And, it seems it''s not something that would be a problem in the future. Keith
At 09:39 PM 12/30/2008, Raghavendra G wrote:>On Tue, Dec 30, 2008 at 12:29 PM, Keith Freedman ><<mailto:freedman at freeformit.com>freedman at freeformit.com> wrote: > > > > so, in the case where they both differ, how will we know which > time will be > > > associated with a file? > > > >The first subvol mentioned in the list is considered, if it is down, > >then the next is considered. > >ok, so I made sure that my subvolume list on the servers is in the same order. >I had always listed the local volume first and the remote one second, >but it sounds like this would lead to each node using it''s own >timestamps or do all the servers in the AFR elect a preferred ''first'' subvol? > > > > and will it fix the on-disk stamp or just ignore it provided the other > > > subvol is online? > > > >It will not fix the time stamp if they differ. > >how about new files being replicated.. they''ll get the timestamp of >the server from which the source file is available? > >in other words.. if I add a new empty volume, and AFR copies >everything to it... what dates will be on the local files? > > >During self-heal afr sets the timestamps (atime and mtime) to that >of the file on the source node.this works fine. to solve my problem, I created a script which gets the file times for each file and creates a set of "touch -d " commands to set the dates on the other server. that was preferable to deleting the directory tree and having things auto-healed since it''s roughly 12 GB''s And, it seems it''s not something that would be a problem in the future. Keith