We are considering using Lustre as our backend file platform. The specific application involves storing a high-volume of sequential data writes, with a moderate amount of reads (mostly sequencial, with some random seeks). Our concern is with reclaiming space. As the file system fills, we need to be able to quickly delete the oldest files. Does Lustre have an efficient file delete? I''m not expecting specific metrics (though they would be nice if available), just some general info about the Lustre delete process (Does it immediately reclaim the space, or do it ''lazily'' in the background? etc, etc.). Thanks.
On Jun 26, 2008, at 1:57 PM, Stew Paddaso wrote:> We are considering using Lustre as our backend file platform. The > specific application involves storing a high-volume of sequential data > writes, with a moderate amount of reads (mostly sequencial, with some > random seeks). Our concern is with reclaiming space. As the file > system fills, we need to be able to quickly delete the oldest files. > > Does Lustre have an efficient file delete? I''m not expecting specific > metrics (though they would be nice if available), just some general > info about the Lustre delete process (Does it immediately reclaim the > space, or do it ''lazily'' in the background? etc, etc.). >I don''t know specifics about if space reclaiming is ''lazy'' or not. But from what I have seen compared to regular ext3 deleting large files on lustre was very fast. I expect this to be because ldsikfs is extent based and regular ext3 is not. If I am wrong on this someone please correct me I really would like to know this also. For me deleting large number of files _feels_ very quick compared to our NFS bobcat from onstor also. Even a operation like the following was much quicker (I wish there was a better way to do this) du -h --max-depth=1> Thanks. > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss > >
On Jun 26, 2008 10:57 -0700, Stew Paddaso wrote:> We are considering using Lustre as our backend file platform. The > specific application involves storing a high-volume of sequential data > writes, with a moderate amount of reads (mostly sequencial, with some > random seeks). Our concern is with reclaiming space. As the file > system fills, we need to be able to quickly delete the oldest files. > > Does Lustre have an efficient file delete? I''m not expecting specific > metrics (though they would be nice if available), just some general > info about the Lustre delete process (Does it immediately reclaim the > space, or do it ''lazily'' in the background? etc, etc.).Well, there is some TINY amount of "laziness" involved, in the sense that the RPC to reclaim the space is sent immediately after getting the approval of the MDS, but it doesn''t wait for any reply. A single client can have up to 8 (by default) destroy RPCs in flight per OST, which we throttle to avoid overloading the server if this is happening on many clients at the same time. There are distributed transactions that ensure the space is freed even in the case of a crash. I don''t think there is any performance problem with Lustre file deletes, in fact a few incidences of "runaway cleanup script" ended up deleting files much more quickly than the site wanted... Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc.
On Sat, 2008-06-28 at 12:19 -0600, Andreas Dilger wrote:> A single > client can have up to 8 (by default) destroy RPCs in flight per OST, > which we throttle to avoid overloading the server if this is happening > on many clients at the same time.It is worth noting that there was a bug open during the 1.6.3ish timeframe where this limitation was not imposed, and in the case of massive deletes in could completely swamp an OSS. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080628/0ff409c5/attachment.bin