Ing. Luca Lazzeroni - Trend Servizi Srl
2018-Jan-16 08:00 UTC
[Gluster-users] Problem with Gluster 3.12.4, VM and sharding
Hi to everyone. I've got a strange problem with a gluster setup: 3 nodes with Centos 7.4, Gluster 3.12.4 from Centos/Gluster repositories, QEMU-KVM version 2.9.0 (compiled from RHEL sources). I'm running volumes in replica 3 arbiter 1 mode (but I've got a volume in "pure" replica 3 mode too). I've applied the "virt" group settings to my volumes since they host VM images. If I try to install something (eg: Ubuntu Server 16.04.3) on a VM (and so I generate a bit of I/O inside it) and configure KVM to access gluster volume directly (via libvirt), install fails after a while because the disk content is corrupted. If I inspect the block inside the disk (by accessing the image directly from outside) I can found many files filled with "^@". If, instead, I configure KVM to access VM images via a FUSE mount, everything seems to work correctly. Note that the problem with install is verified 100% time with QCOW2 image, while it appears only after with RAW disk images. Is there anyone who experienced the same problem ? Thank you, -- Ing. Luca Lazzeroni Responsabile Ricerca e Sviluppo Trend Servizi Srl Tel: 0376/631761 Web: https://www.trendservizi.it
Krutika Dhananjay
2018-Jan-16 10:33 UTC
[Gluster-users] Problem with Gluster 3.12.4, VM and sharding
Please share the volume-info output and the logs under /var/log/glusterfs/ from all your nodes. for investigating the issue. -Krutika On Tue, Jan 16, 2018 at 1:30 PM, Ing. Luca Lazzeroni - Trend Servizi Srl < luca at gvnet.it> wrote:> Hi to everyone. > > I've got a strange problem with a gluster setup: 3 nodes with Centos 7.4, > Gluster 3.12.4 from Centos/Gluster repositories, QEMU-KVM version 2.9.0 > (compiled from RHEL sources). > > I'm running volumes in replica 3 arbiter 1 mode (but I've got a volume in > "pure" replica 3 mode too). I've applied the "virt" group settings to my > volumes since they host VM images. > > If I try to install something (eg: Ubuntu Server 16.04.3) on a VM (and so > I generate a bit of I/O inside it) and configure KVM to access gluster > volume directly (via libvirt), install fails after a while because the disk > content is corrupted. If I inspect the block inside the disk (by accessing > the image directly from outside) I can found many files filled with "^@". > > If, instead, I configure KVM to access VM images via a FUSE mount, > everything seems to work correctly. > > Note that the problem with install is verified 100% time with QCOW2 image, > while it appears only after with RAW disk images. > > Is there anyone who experienced the same problem ? > > Thank you, > > > -- > Ing. Luca Lazzeroni > Responsabile Ricerca e Sviluppo > Trend Servizi Srl > Tel: 0376/631761 > Web: https://www.trendservizi.it > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180116/3c50fc57/attachment.html>
Krutika Dhananjay
2018-Jan-16 10:40 UTC
[Gluster-users] Problem with Gluster 3.12.4, VM and sharding
Also to help isolate the component, could you answer these: 1. on a different volume with shard not enabled, do you see this issue? 2. on a plain 3-way replicated volume (no arbiter), do you see this issue? On Tue, Jan 16, 2018 at 4:03 PM, Krutika Dhananjay <kdhananj at redhat.com> wrote:> Please share the volume-info output and the logs under /var/log/glusterfs/ > from all your nodes. for investigating the issue. > > -Krutika > > On Tue, Jan 16, 2018 at 1:30 PM, Ing. Luca Lazzeroni - Trend Servizi Srl < > luca at gvnet.it> wrote: > >> Hi to everyone. >> >> I've got a strange problem with a gluster setup: 3 nodes with Centos 7.4, >> Gluster 3.12.4 from Centos/Gluster repositories, QEMU-KVM version 2.9.0 >> (compiled from RHEL sources). >> >> I'm running volumes in replica 3 arbiter 1 mode (but I've got a volume in >> "pure" replica 3 mode too). I've applied the "virt" group settings to my >> volumes since they host VM images. >> >> If I try to install something (eg: Ubuntu Server 16.04.3) on a VM (and so >> I generate a bit of I/O inside it) and configure KVM to access gluster >> volume directly (via libvirt), install fails after a while because the disk >> content is corrupted. If I inspect the block inside the disk (by accessing >> the image directly from outside) I can found many files filled with "^@". >> >Also, what exactly do you mean by accessing the image directly from outside? Was it from the brick directories directly? Was it from the mount point of the volume? Could you elaborate? Which files exactly did you check? -Krutika>> If, instead, I configure KVM to access VM images via a FUSE mount, >> everything seems to work correctly. >> >> Note that the problem with install is verified 100% time with QCOW2 >> image, while it appears only after with RAW disk images. >> >> Is there anyone who experienced the same problem ? >> >> Thank you, >> >> >> -- >> Ing. Luca Lazzeroni >> Responsabile Ricerca e Sviluppo >> Trend Servizi Srl >> Tel: 0376/631761 >> Web: https://www.trendservizi.it >> >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users at gluster.org >> http://lists.gluster.org/mailman/listinfo/gluster-users >> > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180116/c15f998d/attachment.html>
Reasonably Related Threads
- [Possibile SPAM] Re: Problem with Gluster 3.12.4, VM and sharding
- [Possibile SPAM] Re: Problem with Gluster 3.12.4, VM and sharding
- [Possibile SPAM] Re: Problem with Gluster 3.12.4, VM and sharding
- [Possibile SPAM] Re: Problem with Gluster 3.12.4, VM and sharding
- [Possibile SPAM] Re: Problem with Gluster 3.12.4, VM and sharding