2016-07-22 1:54 GMT-04:00 Frank Rothenstein <f.rothenstein at bodden-kliniken.de>:> The point is that even if all other backend storage filesystems do correctly > untill 3.7.11 there was no error on ZFS. Something happened nobody ever > could explain in the release of 3.7.12 that makes FUSE-mount _in ovirt_ (it > partly uses dd with iflag=direct , using iflag=direct yourself gives also > errors on the FUSE-mounts ) unusable. > > So 3.7.11 is the last usable version when using ZFS on bricks, afaik. >Can you please share the exact dd command that causes this problem? Thanks, Vijay
On Fri, Jul 22, 2016 at 8:12 AM, Vijay Bellur <vbellur at redhat.com> wrote:> 2016-07-22 1:54 GMT-04:00 Frank Rothenstein < > f.rothenstein at bodden-kliniken.de>: > > The point is that even if all other backend storage filesystems do > correctly > > untill 3.7.11 there was no error on ZFS. Something happened nobody ever > > could explain in the release of 3.7.12 that makes FUSE-mount _in ovirt_ > (it > > partly uses dd with iflag=direct , using iflag=direct yourself gives > also > > errors on the FUSE-mounts ) unusable. > > > > So 3.7.11 is the last usable version when using ZFS on bricks, afaik. > > > > Can you please share the exact dd command that causes this problem? >I want to say it was this one though my logs for vdsm going back that far have rolled off /usr/bin/dd if=/rhev/data-center/mnt/glusterSD/ccgl1.gl.local:GLUSTER1/7c73a8dd-a72e-4556-ac88-7f6813131e64/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1> > Thanks, > Vijay >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160722/701b3379/attachment.html>
Samuli Heinonen
2016-Jul-22 14:03 UTC
[Gluster-users] [Gluster-devel] 3.7.13 & proxmox/qemu
Here is a quick way how to test this: GlusterFS 3.7.13 volume with default settings with brick on ZFS dataset. gluster-test1 is server and gluster-test2 is client mounting with FUSE. Writing file with oflag=direct is not ok: [root at gluster-test2 gluster]# dd if=/dev/zero of=file oflag=direct count=1 bs=1024000 dd: failed to open ?file?: Invalid argument Enable network.remote-dio on Gluster Volume: [root at gluster-test1 gluster]# gluster volume set gluster network.remote-dio enable volume set: success Writing small file with oflag=direct is ok: [root at gluster-test2 gluster]# dd if=/dev/zero of=file oflag=direct count=1 bs=1024000 1+0 records in 1+0 records out 1024000 bytes (1.0 MB) copied, 0.0103793 s, 98.7 MB/s Writing bigger file with oflag=direct is ok: [root at gluster-test2 gluster]# dd if=/dev/zero of=file3 oflag=direct count=100 bs=1M 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.10583 s, 94.8 MB/s Enable Sharding on Gluster Volume: [root at gluster-test1 gluster]# gluster volume set gluster features.shard enable volume set: success Writing small file with oflag=direct is ok: [root at gluster-test2 gluster]# dd if=/dev/zero of=file3 oflag=direct count=1 bs=1M 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115247 s, 91.0 MB/s Writing bigger file with oflag=direct is not ok: [root at gluster-test2 gluster]# dd if=/dev/zero of=file3 oflag=direct count=100 bs=1M dd: error writing ?file3?: Operation not permitted dd: closing output file ?file3?: Operation not permitted -samuli> On 22 Jul 2016, at 16:12, Vijay Bellur <vbellur at redhat.com> wrote: > > 2016-07-22 1:54 GMT-04:00 Frank Rothenstein <f.rothenstein at bodden-kliniken.de>: >> The point is that even if all other backend storage filesystems do correctly >> untill 3.7.11 there was no error on ZFS. Something happened nobody ever >> could explain in the release of 3.7.12 that makes FUSE-mount _in ovirt_ (it >> partly uses dd with iflag=direct , using iflag=direct yourself gives also >> errors on the FUSE-mounts ) unusable. >> >> So 3.7.11 is the last usable version when using ZFS on bricks, afaik. >> > > Can you please share the exact dd command that causes this problem? > > Thanks, > Vijay > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users