I will have a limited window to migrate files to a new OST. I would like to get as far as I can in the window I have. Is it safe to kill lfs_migrate while it is still running? If so will it leave any ''partial copies'' around? Brock Palen www.umich.edu/~brockp CAEN Advanced Computing brockp at umich.edu (734)936-1985
On Feb 27, 2012, at 2:30 PM, Brock Palen wrote:> I will have a limited window to migrate files to a new OST. I would like to get as far as I can in the window I have. > > Is it safe to kill lfs_migrate while it is still running?Yes.> If so will it leave any ''partial copies'' around?Yes, but only the current file. This is based on my version of lfs_migrate (1.8.4 I believe), it may trap signals and delete files in progress now. A quick look at the script will show if it has these traps. -mb -- +----------------------------------------------- | Michael Barnes | | Thomas Jefferson National Accelerator Facility | Scientific Computing Group | 12000 Jefferson Ave. | Newport News, VA 23606 | (757) 269-7634 +-----------------------------------------------
On 27 Feb 2012, at 19:30, Brock Palen wrote:> I will have a limited window to migrate files to a new OST. I would like to get as far as I can in the window I have. > > Is it safe to kill lfs_migrate while it is still running? > > If so will it leave any ''partial copies'' around?The script will be limited by client bandwidth, if possible you could run multiple instances, each working on a different part of the tree you want copied. I''d also consider mounting the FS as a client on the server which hosts the OST and running it there. Ashley.
On Feb 27, 2012, at 2:49 PM, Ashley Pittman wrote:> > On 27 Feb 2012, at 19:30, Brock Palen wrote: > >> I will have a limited window to migrate files to a new OST. I would like to get as far as I can in the window I have. >> >> Is it safe to kill lfs_migrate while it is still running? >> >> If so will it leave any ''partial copies'' around? > > The script will be limited by client bandwidth, if possible you could run multiple instances, each working on a different part of the tree you want copied.Noted and planned on doing that,> > I''d also consider mounting the FS as a client on the server which hosts the OST and running it there.Wasn''t making a server also a client considered bad juju ? Memory pressure things and panics and other badness. This would be nice because the OSS''s have the biggest network pipes in our setup. BTW I am moving old files from old OST''s to new OST''s to balance them back out. in usage and in age distribution.> > Ashley.
On 27 Feb 2012, at 19:52, Brock Palen wrote:>> I''d also consider mounting the FS as a client on the server which hosts the OST and running it there. > > Wasn''t making a server also a client considered bad juju ? Memory pressure things and panics and other badness. This would be nice because the OSS''s have the biggest network pipes in our setup. BTW I am moving old files from old OST''s to new OST''s to balance them back out. in usage and in age distribution.Yes it is, however for it''s very rare for deadlock to actually happen. I wouldn''t recommend it for production but for a scheduled service interval I wouldn''t expect a problem. Ashley.
On 2012-02-27, at 12:52 PM, Brock Palen wrote:> On Feb 27, 2012, at 2:49 PM, Ashley Pittman wrote: >> On 27 Feb 2012, at 19:30, Brock Palen wrote: >>> I will have a limited window to migrate files to a new OST. I would like to get as far as I can in the window I have. >>> >>> Is it safe to kill lfs_migrate while it is still running? >>> >>> If so will it leave any ''partial copies'' around? >> >> The script will be limited by client bandwidth, if possible you could run multiple instances, each working on a different part of the tree you want copied. > > Noted and planned on doing that,Right. Also, it makes the most sense to limit the migration to large files, so that Lustre is running at peak bandwidth and most effectively balancing the space usage. You could spend a lot more time migrating small files, but not significantly affect the space usage.>> I''d also consider mounting the FS as a client on the server which hosts the OST and running it there. > > Wasn''t making a server also a client considered bad juju ? Memory pressure things and panics and other badness.I don''t think there were ever panics because of this. The worst case is that the client thread would hang, and possibly be evicted. We fixed some problems in this area more recently, so the chance of hitting a problem is fairly small.> This would be nice because the OSS''s have the biggest network pipes in our setup. BTW I am moving old files from old OST''s to new OST''s to balance them back out. in usage and in age distribution.Cheers, Andreas -- Andreas Dilger Whamcloud, Inc. Principal Lustre Engineer http://www.whamcloud.com/