Hi David,
As I'm on gluster I use
glusterfs.x86_64 3.7.12-2.el7 @centos-gluster37
I already tried to revert back to 3.7.11 but somehow this didn't help.?
Do you have more info on aio/caching?
FrankAm Dienstag, den 12.07.2016, 04:25 -0500 schrieb David
Gossage:>
> On Tue, Jul 12, 2016 at 4:06 AM, Frank Rothenstein <f.rothenstein at bod
> den-kliniken.de> wrote:
> > Hi all,
> >
> > I am stuck with a strange problem using ovirt/gluster on
> > ZFSonLinux.
> >
> > I detached a data-volume form my Cluster an reimported it to change
> > from POSIX-type/glusterfs to GlusterFS-Storage. When I try to
> > attach it
> > to the Cluster I always get a Sanlock error.?I checked the logs and
> > what I found were some "operation not permitted"
> > errors in the brick logs:
> >
> > [2016-07-08 09:19:24.542046] E [MSGID: 113040] [posix-
> > helpers.c:1684:__posix_fd_ctx_get] 0-datadom-posix: Failed to get
> > anonymous real_path:
> > /tank2/datadom/brick1/br/.glusterfs/c1/43/c1433bab-cd72-4395-ac62-
> > b278670c333a _fd = -1 [No such file or directory]
> > [2016-07-08 09:19:24.542103] W [MSGID: 113006]
> > [posix.c:3010:posix_readv] 0-datadom-posix: pfd is NULL from
> > fd=0x7f15f0003b48 [Operation not permitted]
> > [2016-07-08 09:19:24.542142] E [MSGID: 115068] [server-rpc-
> > fops.c:1486:server_readv_cbk] 0-datadom-server: 4836: READV -2
> > (c1433bab-cd72-4395-ac62-b278670c333a) ==> (Operation not
> > permitted)
> > [Operation not permitted]
> >
> > Anyone ever expierienced such problem?
> >
> > In fact I am able to create/change/delete files in the brick path
> > itself and the mount points (NFS/Fuse).
> > Ovirt itself seems to test the volumes with /usr/bin/dd, but uses
> > iflag=direct. Using this parameter I'm also unable to create any
> > file -
> > is there any gluster-option that changed recently? Or is it ZFS,
> > that
> > introduced something that blocks now?
>
> What version of Gluster are you on?
>
> My errors aren't quite the same as yours, but I have a similar setup
>
> Ovirt 3.6.6/Gluster3.7.11 on ZFSonLinux
>
> Updating from 3.7.11 -> 12 or 13 this last weekend I had these issues
> pop up for me which while not exactly the same did result
> in?[Operation not permitted]
>
> [2016-07-09 15:27:46.935694] I [fuse-bridge.c:4083:fuse_init] 0-
> glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.22
> kernel 7.22
> [2016-07-09 15:27:49.555466] W [MSGID: 114031] [client-rpc-
> fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-1: remote
> operation failed [Operation not permitted]
> [2016-07-09 15:27:49.556574] W [MSGID: 114031] [client-rpc-
> fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-0: remote
> operation failed [Operation not permitted]
> [2016-07-09 15:27:49.556659] W [fuse-bridge.c:2227:fuse_readv_cbk] 0-
> glusterfs-fuse: 80: READ => -1 gfid=deb61291-5176-4b81-8315-
> 3f1cf8e3534d fd=0x7f5224002f68 (Operation not permitted)
> [2016-07-09 15:27:59.612477] W [MSGID: 114031] [client-rpc-
> fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-1: remote
> operation failed [Operation not permitted]
> [2016-07-09 15:27:59.613700] W [MSGID: 114031] [client-rpc-
> fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-0: remote
> operation failed [Operation not permitted]
> [2016-07-09 15:27:59.613781] W [fuse-bridge.c:2227:fuse_readv_cbk] 0-
> glusterfs-fuse: 168: READ => -1 gfid=deb61291-5176-4b81-8315-
> 3f1cf8e3534d fd=0x7f5224002f68 (Operation not permitted)?
>
> Another gluster user suggested that in 3.7.12/13 aio support was
> disabled and I may need to change disk caching method.? So far I have
> just stayed at 3.7.11 until I have time to get a better test setup.
>
> > Thanks in advance, Frank
> >
> >
> >
> >
> >
> > ___________________________________________________________________
> > ___________
> > BODDEN-KLINIKEN Ribnitz-Damgarten GmbH
> > Sandhufe 2
> > 18311 Ribnitz-Damgarten
> >
> > Telefon: 03821-700-0
> > Fax:? ? ? ?03821-700-240
> >
> > E-Mail: info at bodden-kliniken.de? ?Internet: http://www.bodden-klini
> > ken.de
> >
> > Sitz: Ribnitz-Damgarten, Amtsgericht: Stralsund, HRB 2919, Steuer-
> > Nr.: 079/133/40188
> > Aufsichtsratsvorsitzende: Carmen Schr?ter, Gesch?ftsf?hrer: Dr.
> > Falko Milski
> >
> > Der Inhalt dieser E-Mail ist ausschlie?lich f?r den bezeichneten
> > Adressaten bestimmt. Wenn Sie nicht der vorge-
> > sehene Adressat dieser E-Mail oder dessen Vertreter sein sollten,
> > beachten Sie bitte, dass jede Form der Ver?f-
> > fentlichung, Vervielf?ltigung oder Weitergabe des Inhalts dieser E-
> > Mail unzul?ssig ist. Wir bitten Sie, sofort den
> > Absender zu informieren und die E-Mail zu l?schen.
> >
> >
> > ? ? ? ? ? ? ?Bodden-Kliniken Ribnitz-Damgarten GmbH 2016
> > *** Virenfrei durch Kerio Mail Server und Sophos Antivirus ***
> >
> > _______________________________________________
> > Gluster-users mailing list
> > Gluster-users at gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-users
>
______________________________________________________________________________
BODDEN-KLINIKEN Ribnitz-Damgarten GmbH
Sandhufe 2
18311 Ribnitz-Damgarten
Telefon: 03821-700-0
Fax: 03821-700-240
E-Mail: info at bodden-kliniken.de Internet: http://www.bodden-kliniken.de
Sitz: Ribnitz-Damgarten, Amtsgericht: Stralsund, HRB 2919, Steuer-Nr.:
079/133/40188
Aufsichtsratsvorsitzende: Carmen Schr?ter, Gesch?ftsf?hrer: Dr. Falko Milski
Der Inhalt dieser E-Mail ist ausschlie?lich f?r den bezeichneten Adressaten
bestimmt. Wenn Sie nicht der vorge-
sehene Adressat dieser E-Mail oder dessen Vertreter sein sollten, beachten Sie
bitte, dass jede Form der Ver?f-
fentlichung, Vervielf?ltigung oder Weitergabe des Inhalts dieser E-Mail
unzul?ssig ist. Wir bitten Sie, sofort den
Absender zu informieren und die E-Mail zu l?schen.
Bodden-Kliniken Ribnitz-Damgarten GmbH 2016
*** Virenfrei durch Kerio Mail Server und Sophos Antivirus ***