Gluster users - Aug 2020 - gluster over vdo, problem with gfapi

I am not sure that it  is ok to use any caching (at least ovirt  doesn't
uses) .

Have you set the 'virt' group of  settings ? They seem to be optimal ,
but keep in mind that  if you enable them -> you will enable sharding  which
cannot be 'disabled' afterwards.

The fact that it works on C7 is strange,  with wifh version of gluster did you
test.

Best Regards,
Strahil Nikolov

?? 12 ?????? 2020 ?. 18:03:29 GMT+03:00, Dmitry Melekhov <dm at
belkam.com> ??????:
>
>12.08.2020 17:50, Strahil Nikolov ?????:
>> Libgfapi brings far better performance ,
>
>Yes, and several vms do not rely on the same mount point...
>
>
>> but qemu has some limitations.
>>
>>
>> If it works on FUSE , but not on libgfapi -> it seems obvious.
>
>
>Not obvious for me, we tested vdo locally, i.e. without gluster and
>qemu 
>works with cache=none or cache=directsync without problems,
>
>so problem is somewhere in gluster.
>
>>
>> Have you tried to connect from C7 to the Gluster TSP via libgfapi.
>No, but we tested the same setup with gluster 7 with the same result 
>before we upgraded to 8.
>>
>> Also,  is SELINUX in enforcing or not ?
>
>selinux is disabled...
>
>
>Thank you!
>
>>
>> Best Regards,
>> Strahil Nikolov
>>
>> ?? 12 ?????? 2020 ?. 16:34:26 GMT+03:00, Satheesaran Sundaramoorthi
><sasundar at redhat.com> ??????:
>>> On Wed, Aug 12, 2020 at 2:30 PM Dmitry Melekhov <dm at
belkam.com>
>wrote:
>>>
>>>> 12.08.2020 12:55, Amar Tumballi ?????:
>>>>> Hi Dimitry,
>>>>>
>>>>> Was this working earlier and now failing on Version 8 or is
this a
>>> new
>>>>> setup which you did first time?
>>>>>
>>>> Hello!
>>>>
>>>>
>>>> This is first time we  are testing gluster over vdo.
>>>>
>>>> Thank you!
>>>>
>>>>
>>>> Hello Dmitry,
>>> I have been testing the RHEL downstream variant of gluster with
RHEL
>>> 8.2,
>>> where VMs are created with their images on fuse mounted gluster
>volume
>>> with
>>> VDO.
>>> This worked good.
>>>
>>> But I see you are using 'gfapi', so that could be
different.
>>> Though I don't have valuable inputs to help you, do you see
'gfapi'
>>> good
>>> enough than using fuse mounted volume
>
>
>We think that gfapi is better for 2 reasons:
>
>1. it is faster;
>
>2. each qemu process connects to gluster cluster , so there is no one 
>point of failure- fuse mount...
>
>
>Thank you!
>
>>>
>>> -- Satheesaran S

12.08.2020 23:25, Strahil Nikolov ?????:
> I am not sure that it  is ok to use any caching (at least ovirt 
doesn't uses) .
>
> Have you set the 'virt' group of  settings ? They seem to be
optimal , but keep in mind that  if you enable them -> you will enable
sharding  which cannot be 'disabled' afterwards.

Sorry, I don't follow, as I said everything works until we set 
cache=none or cache=directsync in libvirt,

i.e. there is no relation with other gluster settings.
>
> The fact that it works on C7 is strange,  with wifh version of gluster did
you test.
>
>
Dunno, we run qemu on the same host as gluster itself, so we have the 
same gfapi version as gluster server.