Lustre discuss - Jul 2008 - "ls -l" command in the Lustre Filesystem

I found that executing "ls" command would need more time
in the Lustre filesystem than in the ext3 filesystem.
I found a paper that say the performance of the "ls" command in the
ppt of the http://wiki.lustre.org/ ago. But I don''t find it now.
Does its limit be solved. Please tell me where is the paper. Thank you!

On Mon, 2008-07-07 at 03:29 -0700, Johnlya wrote:
> I found that executing "ls" command would need more time
> in the Lustre filesystem than in the ext3 filesystem.
This is not terribly surprising if you think about what Lustre is doing.

An ls on ext3 involves lots of lookup(s) of (meta)data to a very local
device, namely a disk on a SATA/SCSI/IDE bus local in the machine.  The
latency of this lookup is very low.  Those same ls lookups on Lustre has
to go out on a network to another machine to do the same metadata
lookups plus it has to go to the OST(s) that hold the file to get the
size and has to return that data all on the network again, with higher
latencies.

Indeed, there are features in Lustre, such as "statahead" to try to
streamline the data lookups and data movement and reduce the round-trip
latencies, but you are still not working with "like" technologies.

Instead you are paying a small price for huge gains.  For people who
need Lustre, using a local ext3 filesystem simply won''t scale for them.
Lustre will.  Their workload is not usually "ls" either though and
they
appreciate the scalability they get for the small cost of a slower
"ls".

That said, the future will bring clustered metadata (CMD) storage, where
an ls of a large directory could actually be faster than an ls to the
same sized directory on a local ext3 filesystem due to parallelizing
metadata the way we can with object data over many OSTs today.

b.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080708/c910166a/attachment.bin

Thank you!

On Tue 2008-07-08, at 08:16 PM, "Brian J. Murrell"
<Brian.Murr... at Sun.COM> wrote:
> On Mon, 2008-07-07 at 03:29 -0700, Johnlya wrote:
> > I found that executing "ls" command would need more time
> > in the Lustre filesystem than in the ext3 filesystem.
>
> This is not terribly surprising if you think about what Lustre is doing.
>
> An ls on ext3 involves lots of lookup(s) of (meta)data to a very local
> device, namely a disk on a SATA/SCSI/IDE bus local in the machine. ?The
> latency of this lookup is very low. ?Those same ls lookups on Lustre has
> to go out on a network to another machine to do the same metadata
> lookups plus it has to go to the OST(s) that hold the file to get the
> size and has to return that data all on the network again, with higher
> latencies.
>
> Indeed, there are features in Lustre, such as "statahead" to try
to
> streamline the data lookups and data movement and reduce the round-trip
> latencies, but you are still not working with "like"
technologies.
>
> Instead you are paying a small price for huge gains. ?For people who
> need Lustre, using a local ext3 filesystem simply won''t scale for
them.
> Lustre will. ?Their workload is not usually "ls" either though
and they
> appreciate the scalability they get for the small cost of a slower
"ls".
>
> That said, the future will bring clustered metadata (CMD) storage, where
> an ls of a large directory could actually be faster than an ls to the
> same sized directory on a local ext3 filesystem due to parallelizing
> metadata the way we can with object data over many OSTs today.
>
> b.
>
> ?signature.asc
> 1Kdownload
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-disc... at
lists.lustre.orghttp://lists.lustre.org/mailman/listinfo/lustre-discuss

How does "ls" command work? Thank you!

On Wed, 2008-07-08, at 08:16 PM, "Brian J. Murrell"
<Brian.Murr... at Sun.COM> wrote:
> On Mon, 2008-07-07 at 03:29 -0700, Johnlya wrote:
> > I found that executing "ls" command would need more time
> > in the Lustre filesystem than in the ext3 filesystem.
>
> This is not terribly surprising if you think about what Lustre is doing.
>
> An ls on ext3 involves lots of lookup(s) of (meta)data to a very local
> device, namely a disk on a SATA/SCSI/IDE bus local in the machine.  The
> latency of this lookup is very low.  Those same ls lookups on Lustre has
> to go out on a network to another machine to do the same metadata
> lookups plus it has to go to the OST(s) that hold the file to get the
> size and has to return that data all on the network again, with higher
> latencies.
>
> Indeed, there are features in Lustre, such as "statahead" to try
to
> streamline the data lookups and data movement and reduce the round-trip
> latencies, but you are still not working with "like"
technologies.
>
> Instead you are paying a small price for huge gains.  For people who
> need Lustre, using a local ext3 filesystem simply won''t scale for
them.
> Lustre will.  Their workload is not usually "ls" either though
and they
> appreciate the scalability they get for the small cost of a slower
"ls".
>
> That said, the future will bring clustered metadata (CMD) storage, where
> an ls of a large directory could actually be faster than an ls to the
> same sized directory on a local ext3 filesystem due to parallelizing
> metadata the way we can with object data over many OSTs today.
>
> b.
>
>  signature.asc
> 1K??
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-disc... at
lists.lustre.orghttp://lists.lustre.org/mailman/listinfo/lustre-discuss

On Wed, 2008-07-16 at 05:39 -0700, Johnlya wrote:
> How does "ls" command work? Thank you!
That''s beyond the scope of this list.  You could simply download the
source for ls (coreutils on Ubuntu/Debian) and take a look for yourself.
The source will give you a much more comprehensive answer than I can
here.

b.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080717/fe970309/attachment.bin

Thank you!
How to use "stat-ahead" command?

On Thu, 2008-07-17, at 09:37 PM, "Brian J. Murrell"
<Brian.Murr... at Sun.COM> wrote:
> On Wed, 2008-07-16 at 05:39 -0700, Johnlya wrote:
> > How does "ls" command work? Thank you!
>
> That''s beyond the scope of this list. ?You could simply download
the
> source for ls (coreutils on Ubuntu/Debian) and take a look for yourself.
> The source will give you a much more comprehensive answer than I can
> here.
>
> b.
>
> ?signature.asc
> 1Kdownload
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-disc... at
lists.lustre.orghttp://lists.lustre.org/mailman/listinfo/lustre-discuss

Johnlya ??:
> Thank you!
> How to use "stat-ahead" command?
>   
"stat-ahead" feature is enabled by default in lustre 1.6.5 and later 
release.
you can disable it by
echo 0 > /proc/fs/lustre/llite/$client/statahead-max

--
Fan Yong
> On Thu, 2008-07-17, at 09:37 PM, "Brian J. Murrell"
> <Brian.Murr... at Sun.COM> wrote:
>   
>> On Wed, 2008-07-16 at 05:39 -0700, Johnlya wrote:
>>     
>>> How does "ls" command work? Thank you!
>>>       
>> That''s beyond the scope of this list.  You could simply
download the
>> source for ls (coreutils on Ubuntu/Debian) and take a look for
yourself.
>> The source will give you a much more comprehensive answer than I can
>> here.
>>
>> b.
>>
>>  signature.asc
>> 1Kdownload
>>
>> _______________________________________________
>> Lustre-discuss mailing list
>> Lustre-disc... at
lists.lustre.orghttp://lists.lustre.org/mailman/listinfo/lustre-discuss
>>     
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>

> you can disable it by
> echo 0 > /proc/fs/lustre/llite/$client/statahead-max
I want to list files or directories of Lustre Filesystem quickly.
Does it ok after disabling it?
Thank you!

On Thu, 2008-07-24, at 05:18 PM, Yong Fan <Yong.... at Sun.COM>
wrote:
> Johnlya Write:> Thank you!
> > How to use "stat-ahead" command?
>
> "stat-ahead" feature is enabled by default in lustre 1.6.5 and
later
> release.
> you can disable it by
> echo 0 > /proc/fs/lustre/llite/$client/statahead-max
>
> --
> Fan Yong
>
>
>
>
>
> > On Thu, 2008-07-17, at 09:37 PM, "Brian J. Murrell"
> > <Brian.Murr... at Sun.COM> wrote:
>
> >> On Wed, 2008-07-16 at 05:39 -0700, Johnlya wrote:
>
> >>> How does "ls" command work? Thank you!
>
> >> That''s beyond the scope of this list. ?You could simply
download the
> >> source for ls (coreutils on Ubuntu/Debian) and take a look for
yourself.
> >> The source will give you a much more comprehensive answer than I
can
> >> here.
>
> >> b.
>
> >> ?signature.asc
> >> 1Kdownload
>
> >> _______________________________________________
> >> Lustre-discuss mailing list
> >> Lustre-disc... at
lists.lustre.orghttp://lists.lustre.org/mailman/listinfo/lustre-discuss
>
> > _______________________________________________
> > Lustre-discuss mailing list
> > Lustre-disc... at lists.lustre.org
> >http://lists.lustre.org/mailman/listinfo/lustre-discuss
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-disc... at
lists.lustre.orghttp://lists.lustre.org/mailman/listinfo/lustre-discuss-

Johnlya ??:
>> you can disable it by
>> echo 0 > /proc/fs/lustre/llite/$client/statahead-max
>>     
> I want to list files or directories of Lustre Filesystem quickly.
> Does it ok after disabling it?
>   
Dir statahead is just for accelerating "ls -l". If you disable such 
feature, you will benefit nothing from it.

--
Fan Yong
> Thank you!
>
> On Thu, 2008-07-24, at 05:18 PM, Yong Fan <Yong.... at Sun.COM>
wrote:
>   
>> Johnlya Write:> Thank you!
>>     
>>> How to use "stat-ahead" command?
>>>       
>> "stat-ahead" feature is enabled by default in lustre 1.6.5
and later
>> release.
>> you can disable it by
>> echo 0 > /proc/fs/lustre/llite/$client/statahead-max
>>
>> --
>> Fan Yong
>>
>>
>>
>>
>>
>>     
>>> On Thu, 2008-07-17, at 09:37 PM, "Brian J. Murrell"
>>> <Brian.Murr... at Sun.COM> wrote:
>>>       
>>>> On Wed, 2008-07-16 at 05:39 -0700, Johnlya wrote:
>>>>         
>>>>> How does "ls" command work? Thank you!
>>>>>           
>>>> That''s beyond the scope of this list.  You could
simply download the
>>>> source for ls (coreutils on Ubuntu/Debian) and take a look for
yourself.
>>>> The source will give you a much more comprehensive answer than
I can
>>>> here.
>>>>         
>>>> b.
>>>>         
>>>>  signature.asc
>>>> 1Kdownload
>>>>         
>>>> _______________________________________________
>>>> Lustre-discuss mailing list
>>>> Lustre-disc... at
lists.lustre.orghttp://lists.lustre.org/mailman/listinfo/lustre-discuss
>>>>         
>>> _______________________________________________
>>> Lustre-discuss mailing list
>>> Lustre-disc... at lists.lustre.org
>>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>>>       
>> _______________________________________________
>> Lustre-discuss mailing list
>> Lustre-disc... at
lists.lustre.orghttp://lists.lustre.org/mailman/listinfo/lustre-discuss-
>>     
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>