Gluster users - Mar 2017 - Could we not always call lookup for root_dir in fuse_getattr to improve performance?

Went through the code base again. Yes, as you said, we can get rid of
fuse_getattr() not checking nodeid==1 path.

Patch here:  https://review.gluster.org/16944

-Amar



On Fri, Mar 24, 2017 at 2:57 AM, Zhitao Li <zhitaoli1201 at outlook.com>
wrote:
> Hello, everyone!
>
>
> I am now optimizing the performance of "ls". When there are many
little
> files directly in mount point(root dir of glusterfs), I find that
> fuse_getattr takes near half time of total "ls". Strictly, the
nodeid==1
> check in fuse_getattr will call look operation instead of stat, and lookup
> will always miss the md-cache, so it will do real lookup and cost about 3ms
> a time in my case.
>
>
> I doubt whether the special check of nodeid==1 is necessary. I disable the
> check,  it works normal for ls. However, in the "tests" of
gluster, it
> fails(quorum.t). In that case, lookup in root_dir is essential.
>
>
> Up to now, we know that lookup bring high cost in fuse_getattr and it is
> essential in some case. Could anyone give some advice to improve
> fuse_getattr(lookup in root_dir) without bringing bugs?
>
>
> I have been trying to deal  with this for many days. Your help is greatly
> in need.
>
>
>
> Best regards,
> Zhitao Li
>


-- 
Amar Tumballi (amarts)
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20170324/ab6803e1/attachment.html>

Dear Amar,


Thanks very much for your kind help!



I have seen the patch you submit.


I disable the check and do regression test locally. It indeed brings a bug in
"tests/basic/afr/quorum.t"(for tests/ec/, there exists failure but it
is hard to reappear the failure).


For quorum test, it fails  in line_no 36. The case is that for a volume with
replica 2, a brick is killed and quorum-reads is on, quorum-type is fixed. Stat
on the root_dir will fail because of quorum check(afr/afr-read-txn.c). However,
lookup will succeed without quorum check. This is what I have got.

So the solution maybe more complex than removing the nodeid==1 check directly.


Thank you!


Best regards,
Zhitao Li
________________________________
From: Amar Tumballi <atumball at redhat.com>
Sent: Friday, March 24, 2017 9:13:35 PM
To: Zhitao Li
Cc: Niels de Vos; Poornima Gurusiddaiah; Gluster Devel; gluster-users at
gluster.org; Zhitao Li; Zhitao Li
Subject: Re: Could we not always call lookup for root_dir in fuse_getattr to
improve performance?

Went through the code base again. Yes, as you said, we can get rid of
fuse_getattr() not checking nodeid==1 path.

Patch here:  https://review.gluster.org/16944

-Amar



On Fri, Mar 24, 2017 at 2:57 AM, Zhitao Li <zhitaoli1201 at
outlook.com<mailto:zhitaoli1201 at outlook.com>> wrote:

Hello, everyone!


I am now optimizing the performance of "ls". When there are many
little files directly in mount point(root dir of glusterfs), I find that
fuse_getattr takes near half time of total "ls". Strictly, the
nodeid==1 check in fuse_getattr will call look operation instead of stat, and
lookup will always miss the md-cache, so it will do real lookup and cost about
3ms a time in my case.


I doubt whether the special check of nodeid==1 is necessary. I disable the
check,  it works normal for ls. However, in the "tests" of gluster, it
fails(quorum.t). In that case, lookup in root_dir is essential.


Up to now, we know that lookup bring high cost in fuse_getattr and it is
essential in some case. Could anyone give some advice to improve
fuse_getattr(lookup in root_dir) without bringing bugs?


I have been trying to deal  with this for many days. Your help is greatly in
need.



Best regards,
Zhitao Li



--
Amar Tumballi (amarts)
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20170324/9a83aba1/attachment.html>

Test this patch https://review.gluster.org/#/c/16945/

On Fri, Mar 24, 2017 at 9:42 AM, Zhitao Li <zhitaoli1201 at outlook.com>
wrote:
> Dear Amar,
>
>
> Thanks very much for your kind help!
>
>
>
> I have seen the patch you submit.
>
>
> I disable the check and do regression test locally. It indeed *brings a
> bug in "tests/basic/afr/quorum.t"*(for tests/ec/, there exists
failure
> but it is hard to reappear the failure).
>
>
> For quorum test, it fails  in line_no 36. The case is that for a volume
> with replica 2, a brick is killed and quorum-reads is on, quorum-type is
> fixed. Stat on the root_dir will fail because of quorum
> check(afr/afr-read-txn.c). However, lookup will succeed without
> quorum check. This is what I have got.
>
> So the solution maybe more complex than removing the nodeid==1 check
> directly.
>
>
> Thank you!
>
> Best regards,
> Zhitao Li
> ------------------------------
> *From:* Amar Tumballi <atumball at redhat.com>
> *Sent:* Friday, March 24, 2017 9:13:35 PM
> *To:* Zhitao Li
> *Cc:* Niels de Vos; Poornima Gurusiddaiah; Gluster Devel;
> gluster-users at gluster.org; Zhitao Li; Zhitao Li
> *Subject:* Re: Could we not always call lookup for root_dir in
> fuse_getattr to improve performance?
>
> Went through the code base again. Yes, as you said, we can get rid of
> fuse_getattr() not checking nodeid==1 path.
>
> Patch here:  https://review.gluster.org/16944
>
> -Amar
>
>
>
> On Fri, Mar 24, 2017 at 2:57 AM, Zhitao Li <zhitaoli1201 at
outlook.com>
> wrote:
>
>> Hello, everyone!
>>
>>
>> I am now optimizing the performance of "ls". When there are
many little
>> files directly in mount point(root dir of glusterfs), I find that
>> fuse_getattr takes near half time of total "ls". Strictly,
the nodeid==1
>> check in fuse_getattr will call look operation instead of stat, and
lookup
>> will always miss the md-cache, so it will do real lookup and cost about
3ms
>> a time in my case.
>>
>>
>> I doubt whether the special check of nodeid==1 is necessary. I disable
>> the check,  it works normal for ls. However, in the "tests"
of gluster, it
>> fails(quorum.t). In that case, lookup in root_dir is essential.
>>
>>
>> Up to now, we know that lookup bring high cost in fuse_getattr and it
is
>> essential in some case. Could anyone give some advice to improve
>> fuse_getattr(lookup in root_dir) without bringing bugs?
>>
>>
>> I have been trying to deal  with this for many days. Your help is
greatly
>> in need.
>>
>>
>>
>> Best regards,
>> Zhitao Li
>>
>
>
>
> --
> Amar Tumballi (amarts)
>


-- 
Amar Tumballi (amarts)
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20170324/b24ce7e5/attachment.html>