Lustre discuss - May 2013 - replayable IO logging from Lustre's changelog

Hi,

Having read the wiki page on Lustre''s changelogs [1], I have a
question about one claim "Use changelog entries to exactly replicate
changes in a file system mirror."

Of the record types that the changelog supports, there is not an entry
for WRITE, although one can see when files are created as CREAT is
supported. So whilst the example that page allows me to establish that
a file /mydir/hosts exists from 19:10:37 on 2010.03.24 , I know
nothing about the contents of the file `hosts`.

In order to implement the use case "Use changelog entries to exactly
replicate changes in a file system mirror", you''d need something
more
than just the changelog... something that, for example, scp''d from the
IO server the file /mydir/hosts elsewhere. Is there a common procedure
for indeed doing this in Lustre? Or in fact have I missed something in
the wiki page? Are the plans to support WRITE events in Lustre''s
changelogs? I''m not sure what that''d look like -
you''d probably not
want to overload a changelog with raw binary data. However, having
read "Towards an I/O Tracing Framework Taxonomy" [2], I see that
//Trace [3] does support replayable IO traces. Is replayable IO
logging something that the Lustre community has, or intends on
supporting in the future?

[1] - http://wiki.lustre.org/manual/LustreManual20_HTML/LustreMonitoring.html
[2] -
http://www.pdsi-scidac.org/SC07/resources/KONWINSKI-PDSI_Paper-Towards_an_IO_Tracing_Framework_Taxonomy.pdf
[3] - http://www.cs.cmu.edu/~droh/papers/trace-fast07.pdf

--
Rob

On 2013-05-20, at 12:26, "Rob Stewart" <robstewart57 at
gmail.com> wrote:
> Having read the wiki page on Lustre''s changelogs [1], I have a
> question about one claim "Use changelog entries to exactly replicate
> changes in a file system mirror."
> 
> Of the record types that the changelog supports, there is not an entry
> for WRITE, although one can see when files are created as CREAT is
> supported. So whilst the example that page allows me to establish that
> a file /mydir/hosts exists from 19:10:37 on 2010.03.24 , I know
> nothing about the contents of the file `hosts`.
IIRC (this is not something I''ve looked into closely), the MDS will get
an MTIME or CLOSE record that indicates that the file content was modified.
> In order to implement the use case "Use changelog entries to exactly
> replicate changes in a file system mirror", you''d need
something more
> than just the changelog... something that, for example, scp''d from
the
> IO server the file /mydir/hosts elsewhere.
The content of the file would have to be retrieved from the current file itself,
if it still exists, or it would otherwise be deleted.  This also makes the data
copy much more efficient if it is overwritten multiple times, or is written in
small chunks.
> Is there a common procedure
> for indeed doing this in Lustre?
You should take a look at the lustre_rsync tool, which uses the ChangeLog to
clone a source Lustre filesystem to another filesystem.
> Or in fact have I missed something in
> the wiki page? Are the plans to support WRITE events in Lustre''s
> changelogs? I''m not sure what that''d look like -
you''d probably not
> want to overload a changelog with raw binary data.
No, then we''d need a whole duplicate Lustre filesystem just to store
the ChangeLog for the first one...  Storing data is totally impractical in that
regard.
> However, having
> read "Towards an I/O Tracing Framework Taxonomy" [2], I see that
> //Trace [3] does support replayable IO traces. Is replayable IO
> logging something that the Lustre community has, or intends on
> supporting in the future?
> 
> [1] -
http://wiki.lustre.org/manual/LustreManual20_HTML/LustreMonitoring.html
> [2] -
http://www.pdsi-scidac.org/SC07/resources/KONWINSKI-PDSI_Paper-Towards_an_IO_Tracing_Framework_Taxonomy.pdf
> [3] - http://www.cs.cmu.edu/~droh/papers/trace-fast07.pdf
> 
> --
> Rob
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss