Lustre devel - May 2006 - Fwd: Support for Bug 10445 (ADIO)

lustre-devel,

Wei-heng from Northwestern University has reported a performance  
issue with ADIO. Is there anyone one the list able to help with this  
MPI-IO issue?

There is an existing, public Bugzilla on the issue, if you care to  
comment there:
https://bugzilla.clusterfs.com/show_bug.cgi?id=10445

Cheers,
Peter

Begin forwarded message:
> From: Wei-keng Liao <wkliao@ece.northwestern.edu>
> Date: May 1, 2006 2:38:59 PM EDT
> To: Jeffrey Denworth <jeff@clusterfs.com>
> Subject: Re: Support for Bug 10445
>
> I am working on ADIO layer of ROMIO. I saw some low performance
> results for concurrent I/O to a shared file for some I/O
> benchmarks
> on Lustre. So, I tried to improve it through the coding at ADIO
> layer.
> The problem in this bug is a metadata consistency issue. After a
> write call returns, fstat() and lseek() on the writing
> process do not
> see the update of the file pointer. This can be bad for IO
> middleware
> design.
>
> Wei-keng

Hi all,
> There is an existing, public Bugzilla on the issue, if you care to  
> comment there:
> https://bugzilla.clusterfs.com/show_bug.cgi?id=10445
AFAICS, the testcase is bogus. A synchronization is required before rank 0
writes the first 10 bytes. Since all the tasks open the file with O_TRUNC,
another rank can truncate the file _after_ the write.

Cheers,

Johann

Wei-keng, 
 
Lustre clients caches file data/ metadata and periodically flushes the
cache to MDS/OSS. The client-data will also be flushed when file is
closed or synced or locked/unlocked (fcntl()). 
 
In a parallel I/O environment, when multiple clients work on the same
client, it is better to work on non-overlapping sections of the file for
file consistency. This can be achieved with MPI_File_set_view() or do
MPI_File_seek on non-overlapping file offsets. MPI-IO does not guarantee
strict POSIX semantics unless certain MPI-IO semantics/directives are
invoked. In order to guarantee metadata updates being seen immediately,
enable MPI_File_set_atomicity before performing file operations or
follow the sequence (MPI-IO standard should throw more light). 
 
Process1: 
MPI_File_open() 
MPI_File_write_at() 
MPI_File_close() 
                                   Process 2: 
          MPI_File_open() 
  MPI_File_write_at() 
  MPI_File_close() 
 
(or) 
 
Process1:   Process 2: 
MPI_File_open()   MPI_File_open() 
MPI_File_write_at() 
MPI_File_sync()   MPI_File_sync() 
..                                       MPI_File_write_at() 
..   .. 
MPI_File_close   MPI_File_close() 
 
(or) 
 
After file-open enable file-atomicity. This will reduce performance but
file updates are visible to clients. 
 
Hope this helps, 
-Kums 

>>>Peter Bojanic <pbojanic@clusterfs.com> 05/29/06 10:58 am
>>>
lustre-devel,

Wei-heng from Northwestern University has reported a performance 
issue with ADIO. Is there anyone one the list able to help with this 
MPI-IO issue?

There is an existing, public Bugzilla on the issue, if you care to 
comment there:
https://bugzilla.clusterfs.com/show_bug.cgi?id=10445

Cheers,
Peter

Begin forwarded message:
>From: Wei-keng Liao <wkliao@ece.northwestern.edu>
>Date: May 1, 2006 2:38:59 PM EDT
>To: Jeffrey Denworth <jeff@clusterfs.com>
>Subject: Re: Support for Bug 10445
>
>I am working on ADIO layer of ROMIO. I saw some low performance
>results for concurrent I/O to a shared file for some I/O
>benchmarks
>on Lustre. So, I tried to improve it through the coding at ADIO
>layer.
>The problem in this bug is a metadata consistency issue. After a
>write call returns, fstat() and lseek() on the writing
>process do not
>see the update of the file pointer. This can be bad for IO
>middleware
>design.
>
>Wei-keng


Lustre-devel mailing list
Lustre-devel@clusterfs.com
https://mail.clusterfs.com/mailman/listinfo/lustre-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://mail.clusterfs.com/pipermail/lustre-devel/attachments/20060530/e07e9e41/attachment.html