Hi all, I am currently moving off files of a number of OSTs - some in a machine with a predicted hardware failure, some for decommissioning old hardware etc. I''m deactivating the OSTs on the MDS, then "lfs find --obd OSTXXXX_UUID /dir" to create a list of file to migrate. When finished, the OST partitions are still up to 14% full! Mounting as ldiskfs, I can inspect the data in O/0/d??, which all seem to be valid files, text files readable, ownership and atimes meaningfull etc. Since these OSTs have been deactivated on the clients also, nobody has complained about missing files - although this could just mean that our users keep Terabytes of old junk they don''t access any more. So I wonder where these files belong to or why they are not missing. There were no severe crashes of the system in the recent past which could have caused major memory loss of the MDT. Something to worry about? Global lfs_check necessary? Perhaps I will do some "du" runs. Users want this info anyhow, and if I just missed the files in the "lfs find" run, they should show up as errors here. Regards, Thomas
I had the same issue with Lustre 1.8.4. Wash, rinse, repeat.... In other words, do the lfs_find, do the lfs_migrate, then do the find a second time. That seemed to catch most everything of importance. I haven''t a clue why this should be the case, but it was true on every OST that I migrated last fall. bob On 6/27/2011 6:50 AM, Thomas Roth wrote:> Hi all, > > I am currently moving off files of a number of OSTs - some in a machine with a predicted hardware > failure, some for decommissioning old hardware etc. I''m deactivating the OSTs on the MDS, then "lfs > find --obd OSTXXXX_UUID /dir" to create a list of file to migrate. > When finished, the OST partitions are still up to 14% full! Mounting as ldiskfs, I can inspect the > data in O/0/d??, which all seem to be valid files, text files readable, ownership and atimes > meaningfull etc. > Since these OSTs have been deactivated on the clients also, nobody has complained about missing files > - although this could just mean that our users keep Terabytes of old junk they don''t access any more. > So I wonder where these files belong to or why they are not missing. There were no severe crashes of > the system in the recent past which could have caused major memory loss of the MDT. > Something to worry about? Global lfs_check necessary? > > Perhaps I will do some "du" runs. Users want this info anyhow, and if I just missed the files in the > "lfs find" run, they should show up as errors here. > > Regards, > Thomas > > > > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss >
I would check if the file list generated by find contains those remaining (after migration) files. If yes, then there is a chance that those files were changed during migration and checksum didn''t match thus they were not migrated. Also there is a chance that the migration script has a problem with parsing names of some files thus fails to transfer them. I guess you could do some debugging of the migration process and narrow down the problem. Best regards Wojciech On 27 June 2011 14:37, Bob Ball <ball at umich.edu> wrote:> I had the same issue with Lustre 1.8.4. Wash, rinse, repeat.... In > other words, do the lfs_find, do the lfs_migrate, then do the find a > second time. That seemed to catch most everything of importance. I > haven''t a clue why this should be the case, but it was true on every OST > that I migrated last fall. > > bob > > On 6/27/2011 6:50 AM, Thomas Roth wrote: > > Hi all, > > > > I am currently moving off files of a number of OSTs - some in a machine > with a predicted hardware > > failure, some for decommissioning old hardware etc. I''m deactivating the > OSTs on the MDS, then "lfs > > find --obd OSTXXXX_UUID /dir" to create a list of file to migrate. > > When finished, the OST partitions are still up to 14% full! Mounting as > ldiskfs, I can inspect the > > data in O/0/d??, which all seem to be valid files, text files readable, > ownership and atimes > > meaningfull etc. > > Since these OSTs have been deactivated on the clients also, nobody has > complained about missing files > > - although this could just mean that our users keep Terabytes of old junk > they don''t access any more. > > So I wonder where these files belong to or why they are not missing. > There were no severe crashes of > > the system in the recent past which could have caused major memory loss > of the MDT. > > Something to worry about? Global lfs_check necessary? > > > > Perhaps I will do some "du" runs. Users want this info anyhow, and if I > just missed the files in the > > "lfs find" run, they should show up as errors here. > > > > Regards, > > Thomas > > > > > > > > > > _______________________________________________ > > Lustre-discuss mailing list > > Lustre-discuss at lists.lustre.org > > http://lists.lustre.org/mailman/listinfo/lustre-discuss > > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20110627/8e41bce7/attachment.html
If you have the ll_display_filter_fid tool from newer Lustre releases it will
print the MDS inode/generation for the specified OST object (when mounted
locally as type ldiskfs), or a newer e2fsprogs "debugfs stat {file}"
will do the same.
On the MDS the "debugfs ncheck" command will print the pathnames of
one or more inode numbers by scanning the filesystem. On 2.x pathnames can be
found directly without searching by specifying the FID to the "lfs
fid2path" command.
Cheers, Andreas
On 2011-06-27, at 8:39 AM, Wojciech Turek <wjt27 at cam.ac.uk> wrote:
> I would check if the file list generated by find contains those remaining
(after migration) files. If yes, then there is a chance that those files were
changed during migration and checksum didn''t match thus they were not
migrated. Also there is a chance that the migration script has a problem with
parsing names of some files thus fails to transfer them. I guess you could do
some debugging of the migration process and narrow down the problem.
>
> Best regards
>
> Wojciech
>
> On 27 June 2011 14:37, Bob Ball <ball at umich.edu> wrote:
> I had the same issue with Lustre 1.8.4. Wash, rinse, repeat.... In
> other words, do the lfs_find, do the lfs_migrate, then do the find a
> second time. That seemed to catch most everything of importance. I
> haven''t a clue why this should be the case, but it was true on
every OST
> that I migrated last fall.
>
> bob
>
> On 6/27/2011 6:50 AM, Thomas Roth wrote:
> > Hi all,
> >
> > I am currently moving off files of a number of OSTs - some in a
machine with a predicted hardware
> > failure, some for decommissioning old hardware etc. I''m
deactivating the OSTs on the MDS, then "lfs
> > find --obd OSTXXXX_UUID /dir" to create a list of file to
migrate.
> > When finished, the OST partitions are still up to 14% full! Mounting
as ldiskfs, I can inspect the
> > data in O/0/d??, which all seem to be valid files, text files
readable, ownership and atimes
> > meaningfull etc.
> > Since these OSTs have been deactivated on the clients also, nobody has
complained about missing files
> > - although this could just mean that our users keep Terabytes of old
junk they don''t access any more.
> > So I wonder where these files belong to or why they are not missing.
There were no severe crashes of
> > the system in the recent past which could have caused major memory
loss of the MDT.
> > Something to worry about? Global lfs_check necessary?
> >
> > Perhaps I will do some "du" runs. Users want this info
anyhow, and if I just missed the files in the
> > "lfs find" run, they should show up as errors here.
> >
> > Regards,
> > Thomas
> >
> >
> >
> >
> > _______________________________________________
> > Lustre-discuss mailing list
> > Lustre-discuss at lists.lustre.org
> > http://lists.lustre.org/mailman/listinfo/lustre-discuss
> >
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20110627/57e32292/attachment-0001.html