Gluster users - Jul 2016 - [Gluster-devel] 3.7.13 & proxmox/qemu

Hi all,

I?m running oVirt 3.6 and Gluster 3.7 with ZFS backend. All hypervisor and
storage nodes have CentOS 7. I was planning to upgrade to 3.7.13 during weekend
but i?ll probably wait for more information on this issue.

Afaik ZFS on Linux doesn?t support aio. Has there been some changes to GlusterFS
regarding aio?

Best regards,
Samuli heinonen
> On 21 Jul 2016, at 21:00, David Gossage <dgossage at
carouselchecks.com> wrote:
> 
> On Thu, Jul 21, 2016 at 12:48 PM, David Gossage <dgossage at
carouselchecks.com> wrote:
> On Thu, Jul 21, 2016 at 9:58 AM, David Gossage <dgossage at
carouselchecks.com> wrote:
> On Thu, Jul 21, 2016 at 9:52 AM, Niels de Vos <ndevos at redhat.com>
wrote:
> On Sun, Jul 10, 2016 at 10:49:52AM +1000, Lindsay Mathieson wrote:
> > Did a quick test this morning - 3.7.13 is now working with libgfapi -
yay!
> >
> >
> > However I do have to enable write-back or write-through caching in
qemu
> > before the vm's will start, I believe this is to do with aio
support. Not a
> > problem for me.
> >
> > I see there are settings for storage.linux-aio and storage.bd-aio -
not sure
> > as to whether they are relevant or which ones to play with.
> 
> Both storage.*-aio options are used by the brick processes. Depending on
> what type of brick you have (linux = filesystem, bd = LVM Volume Group)
> you could enable the one or the other.
> 
> We do have a strong suggestion to set these "gluster volume group
.."
> options:
>  
https://github.com/gluster/glusterfs/blob/master/extras/group-virt.example
> 
> From those options, network.remote-dio seems most related to your aio
> theory. It was introduced with http://review.gluster.org/4460 that
> contains some more details.
> 
> 
> Wonder if this may be related at all
> 
> * #1347553: O_DIRECT support for sharding
> https://bugzilla.redhat.com/show_bug.cgi?id=1347553
> 
> Is it possible to downgrade from 3.8 back to 3.7.x 
> 
> Building test box right now anyway but wondering.
> 
> May be anecdotal with small sample size but the few people who have had
issue all seemed to have zfs backed gluster volumes.
> 
> Now that i recall back to the day I updated.  The gluster volume on xfs I
use for my hosted engine never had issues.
>  
> 
>  
> 
> Thanks with the exception of stat-prefetch I have those enabled 
> I could try turning that back off though at the time of update to 3.7.13 it
was off.  I didnt turn it back on till later in next week after downgrading back
to 3.7.11.
> 
> Number of Bricks: 1 x 3 = 3
> Transport-type: tcp
> Bricks:
> Brick1: ccgl1.gl.local:/gluster1/BRICK1/1
> Brick2: ccgl2.gl.local:/gluster1/BRICK1/1
> Brick3: ccgl4.gl.local:/gluster1/BRICK1/1
> Options Reconfigured:
> diagnostics.brick-log-level: WARNING
> features.shard-block-size: 64MB
> features.shard: on
> performance.readdir-ahead: on
> storage.owner-uid: 36
> storage.owner-gid: 36
> performance.quick-read: off
> performance.read-ahead: off
> performance.io-cache: off
> performance.stat-prefetch: on
> cluster.eager-lock: enable
> network.remote-dio: enable
> cluster.quorum-type: auto
> cluster.server-quorum-type: server
> server.allow-insecure: on
> cluster.self-heal-window-size: 1024
> cluster.background-self-heal-count: 16
> performance.strict-write-ordering: off
> nfs.disable: on
> nfs.addr-namelookup: off
> nfs.enable-ino32: off
> 
> 
> HTH,
> Niels
> 
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
> 
> 
> 
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users

On 07/21/2016 02:38 PM, Samuli Heinonen wrote:
> Hi all,
> 
> I?m running oVirt 3.6 and Gluster 3.7 with ZFS backend. 
> ...
> Afaik ZFS on Linux doesn?t support aio. Has there been some changes to
GlusterFS regarding aio?
> 
Boy, if that isn't a smoking gun, I don't know what is.

--

Kaleb