Miguel Molowny Lopez
2010-Jan-18 16:12 UTC
[Lustre-discuss] e2scan wrong file list mtime/ctime
Hi Andreas, we have reproduced a test case and analyzed it with ''debugfs'' and ''stat'' commands the inconsistencies. As you can see in the attached file we have noticed two kinds of problem, since we see two different behaviors. First of all, we have built two different lists with e2scan: One with files modified from 1 week ago (from Jan 11th 2010) and one with the files modified 3 days ago (Jan 15th 2010). After this, we have sorted both lists and taken the difference (with ''diff'' command) to have as result the set of files modified between Jan 11th 2010 and Jan 14th 2010, both included. We have analyzed with the ''stat'' command the mtime and ctime of the resulting list of files, observing that there were files in it with their mtime and ctime between Jan 15th and Jan 18th, while the output of debugfs command for these files reports dates before Jan 15th. So it seems that we may have some kind of inconsistence in the MDS. What do you think about it? The second problem we noticed is that some files show the same timestamp for mtime, ctime and atime with both commands (for example Jan 17th), but ''e2scan'' shows them in the list of files modified from Jan 11th but not in the list from Jan 15th. The only difference with these files is that ''debugfs'' command shows the crtime which is several months in the past, but this is not relevant for ''e2scan''. Could this be a bug of ''e2scan''? You can see the command output details in the attached file. For each file analyzed you will find a case comment, the ''stat'' command output, a blank line and the ''debugfs'' command output followed by a separation line (===========). Thanks a lot in advance, Miguel Subject: Re: [Lustre-discuss] e2scan wrong file list mtime/ctime From: Andreas Dilger <adilger at sun.com> Date: Thu, 07 Jan 2010 15:59:06 -0700 To: "f4 at caspur.it" <f4 at caspur.it> CC: lustre-discuss at lists.lustre.org On 2010-01-07, at 03:43, Andrea Pieretti wrote: > > sorry for answering you so late (Christmas holidays ;) , > > it seems that the information reported by the two commands you > > indicated, is different regarding the Access time of the file. > > > > Is this a normal behaviour ? The atime is only updated lazily on the MDS. What is important for the e2scan problem you reported is the mtime and ctime, which are identical in this case. Can you please reproduce with the original test case, and then report the timestamps of a file that does not show up in your e2scan list correctly. > > It seems that the MDS does not update the access time (we opened > > the file on the client to view it''s content) > > > > > > ---> Issued on a lustre client # stat /scratch/pieretti/new.out > > File: `/scratch/pieretti/new.out'' > > Size: 25959 Blocks: 56 IO Block: 2097152 regular file > > Device: a2f3dcdch/2733890780d Inode: 64740446 Links: 1 > > Access: (0644/-rw-r--r--) Uid: ( 244/pieretti) Gid: (20053/ > > bb) > > Access: 2010-01-07 11:28:56.000000000 +0100 > > Modify: 2009-04-04 23:44:36.000000000 +0200 > > Change: 2009-06-19 00:35:21.000000000 +0200 > > > > ---> Issued on the mds # debugfs -c -R ''stat <64740446>'' /dev/mapper/ > > scratch_mdt > > debugfs 1.41.6.sun1 (30-May-2009) > > /dev/mapper/scratch_mdt: catastrophic mode - not reading inode or > > group bitmaps > > Inode: 64740446 Type: regular Mode: 0644 Flags: 0x0 > > Generation: 1095573155 Version: 0x00000000:00000000 > > User: 244 Group: 20053 Size: 0 > > File ACL: 0 Directory ACL: 0 > > Links: 1 Blockcount: 0 > > Fragment: Address: 0 Number: 0 Size: 0 > > ctime: 0x4a3ac129:c210ae38 -- Fri Jun 19 00:35:21 2009 > > atime: 0x4a3ac129:00000000 -- Fri Jun 19 00:35:21 2009 > > mtime: 0x49d7d4c4:00000000 -- Sat Apr 4 23:44:36 2009 > > crtime: 0x4a3ac129:c1969bb4 -- Fri Jun 19 00:35:21 2009 > > Size of extra inode fields: 28 > > Extended attributes stored in inode body: > > lov = "d0 0b d1 0b 01 00 00 00 5e dc db 03 00 00 00 00 00 00 00 00 > > 00 00 00 00 00 00 10 00 08 00 > > 00 00 f5 cd 0c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 05 > > 00 00 00 f5 cd 0c 00 00 00 0 > > 0 00 00 00 00 00 00 00 00 00 00 00 00 00 04 00 00 00 f5 cd 0c 00 00 > > 00 00 00 00 00 00 00 00 00 00 > > 00 00 00 00 00 07 00 00 00 f5 cd 0c 00 00 00 00 00 00 00 00 00 00 00 > > 00 00 00 00 00 00 06 00 00 00 > > f5 cd 0c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 > > 00 f5 cd 0c 00 00 00 00 00 0 > > 0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 f5 cd 0c 00 00 00 00 > > 00 00 00 00 00 00 00 00 00 00 > > 00 00 00 03 00 00 00 f5 cd 0c 00 00 00 00 00 00 00 00 00 00 00 00 00 > > 00 00 00 00 02 00 00 00 " (22 > > 4) > > BLOCKS: > > > > > > Thanks for your kindness > > > > Andrea > > > > > > -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 17jan.err Url: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100118/bcd0be60/attachment-0001.pl
Miguel Molowny Lopez
2010-Jan-25 09:44 UTC
[Lustre-discuss] e2scan wrong file list mtime/ctime
Hi Andreas, we have reproduced a test case and analyzed it with ''debugfs'' and ''stat'' commands the inconsistencies. As you can see in the attached file we have noticed two kinds of problem, since we see two different behaviors. First of all, we have built two different lists with e2scan: One with files modified from 1 week ago (from Jan 11th 2010) and one with the files modified 3 days ago (Jan 15th 2010). After this, we have sorted both lists and taken the difference (with ''diff'' command) to have as result the set of files modified between Jan 11th 2010 and Jan 14th 2010, both included. We have analyzed with the ''stat'' command the mtime and ctime of the resulting list of files, observing that there were files in it with their mtime and ctime between Jan 15th and Jan 18th, while the output of debugfs command for these files reports dates before Jan 15th. So it seems that we may have some kind of inconsistence in the MDS. What do you think about it? The second problem we noticed is that some files show the same timestamp for mtime, ctime and atime with both commands (for example Jan 17th), but ''e2scan'' shows them in the list of files modified from Jan 11th but not in the list from Jan 15th. The only difference with these files is that ''debugfs'' command shows the crtime which is several months in the past, but this is not relevant for ''e2scan''. Could this be a bug of ''e2scan''? You can see the command output details in the attached file. For each file analyzed you will find a case comment, the ''stat'' command output, a blank line and the ''debugfs'' command output followed by a separation line (===========). Thanks a lot in advance, Miguel Subject: Re: [Lustre-discuss] e2scan wrong file list mtime/ctime From: Andreas Dilger <adilger at sun.com> Date: Thu, 07 Jan 2010 15:59:06 -0700 To: "f4 at caspur.it" <f4 at caspur.it> CC: lustre-discuss at lists.lustre.org On 2010-01-07, at 03:43, Andrea Pieretti wrote: > > sorry for answering you so late (Christmas holidays ;) , > > it seems that the information reported by the two commands you > > indicated, is different regarding the Access time of the file. > > > > Is this a normal behaviour ? The atime is only updated lazily on the MDS. What is important for the e2scan problem you reported is the mtime and ctime, which are identical in this case. Can you please reproduce with the original test case, and then report the timestamps of a file that does not show up in your e2scan list correctly. > > It seems that the MDS does not update the access time (we opened > > the file on the client to view it''s content) > > > > > > ---> Issued on a lustre client # stat /scratch/pieretti/new.out > > File: `/scratch/pieretti/new.out'' > > Size: 25959 Blocks: 56 IO Block: 2097152 regular file > > Device: a2f3dcdch/2733890780d Inode: 64740446 Links: 1 > > Access: (0644/-rw-r--r--) Uid: ( 244/pieretti) Gid: (20053/ > > bb) > > Access: 2010-01-07 11:28:56.000000000 +0100 > > Modify: 2009-04-04 23:44:36.000000000 +0200 > > Change: 2009-06-19 00:35:21.000000000 +0200 > > > > ---> Issued on the mds # debugfs -c -R ''stat <64740446>'' /dev/mapper/ > > scratch_mdt > > debugfs 1.41.6.sun1 (30-May-2009) > > /dev/mapper/scratch_mdt: catastrophic mode - not reading inode or > > group bitmaps > > Inode: 64740446 Type: regular Mode: 0644 Flags: 0x0 > > Generation: 1095573155 Version: 0x00000000:00000000 > > User: 244 Group: 20053 Size: 0 > > File ACL: 0 Directory ACL: 0 > > Links: 1 Blockcount: 0 > > Fragment: Address: 0 Number: 0 Size: 0 > > ctime: 0x4a3ac129:c210ae38 -- Fri Jun 19 00:35:21 2009 > > atime: 0x4a3ac129:00000000 -- Fri Jun 19 00:35:21 2009 > > mtime: 0x49d7d4c4:00000000 -- Sat Apr 4 23:44:36 2009 > > crtime: 0x4a3ac129:c1969bb4 -- Fri Jun 19 00:35:21 2009 > > Size of extra inode fields: 28 > > Extended attributes stored in inode body: > > lov = "d0 0b d1 0b 01 00 00 00 5e dc db 03 00 00 00 00 00 00 00 00 > > 00 00 00 00 00 00 10 00 08 00 > > 00 00 f5 cd 0c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 05 > > 00 00 00 f5 cd 0c 00 00 00 0 > > 0 00 00 00 00 00 00 00 00 00 00 00 00 00 04 00 00 00 f5 cd 0c 00 00 > > 00 00 00 00 00 00 00 00 00 00 > > 00 00 00 00 00 07 00 00 00 f5 cd 0c 00 00 00 00 00 00 00 00 00 00 00 > > 00 00 00 00 00 00 06 00 00 00 > > f5 cd 0c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 > > 00 f5 cd 0c 00 00 00 00 00 0 > > 0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 f5 cd 0c 00 00 00 00 > > 00 00 00 00 00 00 00 00 00 00 > > 00 00 00 03 00 00 00 f5 cd 0c 00 00 00 00 00 00 00 00 00 00 00 00 00 > > 00 00 00 00 02 00 00 00 " (22 > > 4) > > BLOCKS: > > > > > > Thanks for your kindness > > > > Andrea > > > > > > -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 17jan.err Url: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100125/095487f3/attachment.pl