hi guys
This is the design doc of lustre join file. If you are interested in it,
you can have a look.
thanks
wangdi
--
Cheers
Wangdi
___________________________________________________
Wangdi Cluster File Systems, Inc
http://www.clusterfs.com
Tel:+86-10-6554-1806
Fu Hu Plaza D-12-A
No. 8, Chao Yang Men Bei Da Street Dongcheng District
Beijing, P.R.C. 100027
Tel: +86(10)6554-1806
e-mail:Wang Di <wangdi@clusterfs.com>
___________________________________________________
-------------- next part --------------
A non-text attachment was scrubbed...
Name: join_file.pdf
Type: application/pdf
Size: 300439 bytes
Desc: not available
Url :
http://mail.clusterfs.com/pipermail/lustre-devel/attachments/20060427/d4a95bb7/join_file-0001.pdf
It would be nice to have an executive summary of what a join file is,
and the reasons behind it... It sure would make a whole lot more sense
to readers of the document. IE you would know what a join file is for
instead of trying to figure it out as you read the document.
Evan
-----Original Message-----
From: lustre-devel-bounces@clusterfs.com
[mailto:lustre-devel-bounces@clusterfs.com] On Behalf Of wangdi
Sent: Thursday, April 27, 2006 7:50 AM
To: lustre-devel@clusterfs.com
Subject: [Lustre-devel] lustre join file design doc
hi guys
This is the design doc of lustre join file. If you are interested in it,
you can have a look.
thanks
wangdi
--
Cheers
Wangdi
___________________________________________________
Wangdi Cluster File Systems, Inc
http://www.clusterfs.com
Tel:+86-10-6554-1806
Fu Hu Plaza D-12-A
No. 8, Chao Yang Men Bei Da Street Dongcheng District Beijing, P.R.C.
100027
Tel: +86(10)6554-1806
e-mail:Wang Di <wangdi@clusterfs.com>
___________________________________________________
On Apr 28, 2006 11:01 -0700, Felix, Evan J wrote:> It would be nice to have an executive summary of what a join file is, > and the reasons behind it... It sure would make a whole lot more sense > to readers of the document. IE you would know what a join file is for > instead of trying to figure it out as you read the document.A file join is currently implemented to emulate a feature in a Cray? filesystem, where a series of existing files are "joined" together to produce a single, larger file. This is essentially concatenation done smartly, without moving any data around: "join f1 f2 f3 f4 f5" == "cat f1 f2 f3 f4 f5 > tt; mv tt f1" I personally have little idea of what the demand for this functionality is, as it stands by itself. From a Lustre POV, however, the same functionality could be used to implement a variety of interesting and arbitrary striping patterns, for complex data formats like HDF5 where some parts of the file are accessed by all clients (and want wide striping) and others are file-per-process (and want 1 or 2 stripes). It could also be used as the basis for the case where an existing file is out of space on the OST(s) that it is currently using, so a second set of objects would be "joined" at the end of the file so it could continue to grow. That wouldn''t help the case where a file is sparse and being written in the middle, but that is a less common case. Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc.