Richard Henwood
2012-May-29 15:30 UTC
[Lustre-discuss] [wc-discuss] Tar backup of MDT runs extremely slow, tar pauses on pointers to very large files
On Tue, May 29, 2012 at 7:30 AM, Jeremy Filizetti <jeremy.filizetti at gmail.com> wrote:> Your probably using a version of tar which doesn''t check the number of > blocks used by the file to determine if it is entirely sparse.? Without that > optimization you will read the entire file (of zero fill) to determine if it > is sparse.? GNU tar >= 1.24 seems to have the code for this but lustre-tar > on Whamcloud''s site doesn''t. >Hi All, Whamcloud provide a modified version of tar 1.19 that includes the ''sparse files'' optimization. It has been targeted at RHEL5 but is expected to work on RHEL6 as well. I''ve made some notes on the this work: http://wiki.whamcloud.com/display/PUB/Lustre+Tools#LustreTools-lustretar As you point out, the sparse files optimization is included since gnu tar 1.24 (but, was not documented in the release notes if I remember correctly). I have been talking to RedHat to include the spase files patch and I expect RHEL 6.3 will have a suitable change in tar to achieve this behavior. cheers, Richard <snip> -- Richard.Henwood at whamcloud.com Whamcloud Inc. tel: +1 512 410 9612