Lustre discuss - Jun 2010 - Lustre and loop devices

Hi!
Here
http://blogs.sun.com/atulvid/entry/improving_performance_of_small_files is
described a very interesting way to improve the performance of small files
on a Lustre storage - to mount a loopback file, which is actually placed on
the storage with LustreFS. I checked - it really worked. It greatly improved
the speed. But in the end of the doc there is a notice:
"Under heavy memory loads, Lustre sometime drops pages of IO going to
loopback mounted files. This is a known issue is being worked on."
What does it actually mean? And how it can affect my data on the storage?
And does anybody actually use this method? Is there any way to use it
safely?


-----
WBR, Sergey Arlashin
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100615/1652dff4/attachment.html

See https://bugzilla.lustre.org/show_bug.cgi?id=22004

The work-around is to patch the loopback driver (loop.c) on the client node.

Kevin


Sergey Arlashin wrote:
> Hi!
> Here  
> http://blogs.sun.com/atulvid/entry/improving_performance_of_small_files 
> is described a very interesting way to improve the performance of 
> small files on a Lustre storage - to mount a loopback file, which is 
> actually placed on the storage with LustreFS. I checked - it really 
> worked. It greatly improved the speed. But in the end of the doc there 
> is a notice:
> "Under heavy memory loads, Lustre sometime drops pages of IO going to 
> loopback mounted files. This is a known issue is being worked on."
> What does it actually mean? And how it can affect my data on the storage?
> And does anybody actually use this method? Is there any way to use it 
> safely?
>
>
> -----
> WBR, Sergey Arlashin
> ------------------------------------------------------------------------
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>

If you have a huge number of very small file, is really impressive how 
the loopback device help. But the problem is that you could access only 
from 1 client. The loopback device is a ext2 or ext3 filesystem and you 
can not mount it from more than 1 client!!!

On 06/15/2010 09:53 PM, Kevin Van Maren wrote:
> See https://bugzilla.lustre.org/show_bug.cgi?id=22004
>
> The work-around is to patch the loopback driver (loop.c) on the client
node.
>
> Kevin
>
>
> Sergey Arlashin wrote:
>    
>> Hi!
>> Here
>> http://blogs.sun.com/atulvid/entry/improving_performance_of_small_files
>> is described a very interesting way to improve the performance of
>> small files on a Lustre storage - to mount a loopback file, which is
>> actually placed on the storage with LustreFS. I checked - it really
>> worked. It greatly improved the speed. But in the end of the doc there
>> is a notice:
>> "Under heavy memory loads, Lustre sometime drops pages of IO going
to
>> loopback mounted files. This is a known issue is being worked on."
>> What does it actually mean? And how it can affect my data on the
storage?
>> And does anybody actually use this method? Is there any way to use it
>> safely?
>>
>>
>> -----
>> WBR, Sergey Arlashin
>>
------------------------------------------------------------------------
>>
>> _______________________________________________
>> Lustre-discuss mailing list
>> Lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>>
>>      
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>
>    

-- 
_Gabriele Paciucci_ http://www.linkedin.com/in/paciucci

Pursuant to legislative Decree n. 196/03 you are hereby informed that this email
contains confidential information intended only for use of addressee. If you are
not the addressee and have received this email by mistake, please send this
email to the sender. You may not copy or disseminate this message to anyone.
Thank You.